torben/chemenu
1
1 Fork 0
Code Issues 40 Pull Requests Actions Packages Releases 41 Wiki Activity

feat: docs reference generates a human command reference under docs/reference/ from the command records, one page per group plus exit codes and gates; toc never lands inside another generated region (#188)
CI / verify (push) Successful in 6m11s
Details
CI / pwsh (push) Successful in 2m0s
Details
Release / release (push) Successful in 35s
Details

Browse Source 
Files changed:
- CHANGES.md
- README.md
- VERSION
- docs/reference/README.md
- docs/reference/commands/README.md
- docs/reference/commands/catalog-and-log.md
- docs/reference/commands/content-migrations.md
- docs/reference/commands/distribution-and-versioning.md
- docs/reference/commands/finding-and-checking.md
- docs/reference/commands/git.md
- docs/reference/commands/guideline-export.md
- docs/reference/commands/instance-health.md
- docs/reference/commands/links-and-citations.md
- docs/reference/commands/pages.md
- docs/reference/commands/provenance.md
- docs/reference/commands/raw-material-and-uploads.md
- docs/reference/commands/telemetry.md
- docs/reference/commands/types-instructions-and-docs.md
- docs/reference/commands/workshop-runs-and-session-budget.md
- docs/reference/exit-codes.md
- instructions/dev/doc-pull-through.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/cli.py
- tools/chemenu/cli_contract.py
- tools/chemenu/commands/docs_verify.py
- tools/chemenu/reference_doc.py
- tools/chemenu/tests/test_reference_doc.py
- tools/chemenu/toc.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
torbenandClaude Opus 5.5 committed 2026-10-06 21:29:01 +02:00
1 parent 01c888d3cd
commit 53383e7e9b
29 files changed
+5079 -29
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

CHANGES.md
+13 -1
View File
@@ -14,7 +14,7 @@ place only: `instructions/dev/version-parts.md` § The candidate model.
---
## 9.0.0-beta.2 - 2026-10-06 - AGENTS.md routing line and a toc.py docstring describe docs/ as the whole human documentation
## 9.0.0-beta.3 - 2026-10-06 - docs reference: human command reference under docs/reference/, generated from the command records
**Author:** Torben Nehmer
@@ -28,6 +28,7 @@ place only: `instructions/dev/version-parts.md` § The candidate model.
**Medium impact**
- Changelog kompakt: ein Absatz pro Thema, Größenbudget, Migrationsverweis
- docs reference: human command reference under docs/reference/, generated from the command records
**Low impact**
- version bump: Abschlussmeldung nennt Stichpunkt oder Themenabsatz statt pauschal Prosa
@@ -55,6 +56,17 @@ Migrationsdokument auf die Basis des Kandidaten, schreibt jeder `version bump` d
grenzüberschreitenden Eintrag, der sein Dokument nicht nennt. Die Formregeln stehen nur noch in
`instructions/dev/version-parts.md`.
### Kommandoreferenz für Menschen unter `docs/reference/` (#188)
Der neue Befehl `wikitool docs reference [--apply]` erzeugt aus den Kommando-Datensätzen eine Seite
je Gruppe unter `docs/reference/commands/`, eine Übersicht in deren `README.md` und
`docs/reference/exit-codes.md` mit den Gates und allen Ursachen für Exit 42. Die Seiten zeigen
Synopsis, Optionen, Beispiele, Verhalten, Eigenschaften in Worten und Exit-Codes mit Ursache,
nicht aber die an Agenten gerichteten ON-FAILURE-Reaktionen und NEVER-Zeilen. Titel und Kopfsatz
schreibt ein Mensch; das Werkzeug legt keine Seite an. `docs verify` meldet eine veraltete Region,
eine Gruppe ohne Seite und eine Seite ohne Gruppe, solange es `docs/` gibt. Das Inhaltsverzeichnis
kommt nie mehr in eine andere generierte Region hinein.
---
## 8.0.0 - 2026-10-06 - Installation nur aus Releases, Preflight und native Windows-Unterstützung, incoming/ als Warteschlange
README.md
+2 -1
View File
@@ -66,7 +66,8 @@ The human documentation lives under `docs/`, sorted by what you want from it:
- [docs/how-to/](docs/how-to/README.md) - doing: one task per page, for the operator of an
instance.
- [docs/reference/](docs/reference/README.md) - looking up: commands, exit codes, skills, page
types. The command reference is [tools/CONTRACT.md](tools/CONTRACT.md).
types. The command reference is [docs/reference/commands/](docs/reference/commands/README.md);
agents use the same records in [tools/CONTRACT.md](tools/CONTRACT.md).
- [docs/explanation/](docs/explanation/README.md) - understanding: why the stack is built the way
it is.
VERSION
+1 -1
View File
@@ -1 +1 @@
9.0.0-beta.2
9.0.0-beta.3
docs/reference/README.md
+7 -3
View File
@@ -5,6 +5,10 @@ reference page describes what is, not how to use it or why it is so - and wherev
it exists in the code, it is generated from that source and checked against it rather than
written by hand.
Nothing is generated here yet. Until it is, the command reference is
[tools/CONTRACT.md](../../tools/CONTRACT.md) (and `tools/wikitool <command> -h`), and a page
type is described by `tools/wikitool types describe <type>`.
- [Command reference](commands/README.md) - every `wikitool` command, one page per group,
generated from the same records as `tools/wikitool <command> -h`.
- [Exit codes and gates](exit-codes.md) - which commands can stop and wait for a person, and
why.
Page types are not generated here yet; until they are, a page type is described by
`tools/wikitool types describe <type>`.
docs/reference/commands/README.md
+164
View File
@@ -0,0 +1,164 @@
# Command reference
Every `wikitool` command, one page per group, generated from the same records `tools/wikitool
<command> -h` prints. Each command's section gives its synopsis, options, examples, behaviour,
properties and exit codes; what a command exits 42 for, and which gate it belongs to, is
collected in [exit codes and gates](../exit-codes.md).
<!-- wikitool:toc -->
## Contents
- [Pages](#pages)
- [Links and citations](#links-and-citations)
- [Catalog and log](#catalog-and-log)
- [Finding and checking](#finding-and-checking)
- [Provenance](#provenance)
- [Raw material and uploads](#raw-material-and-uploads)
- [Git](#git)
- [Guideline export](#guideline-export)
- [Workshop runs and session budget](#workshop-runs-and-session-budget)
- [Types, instructions and docs](#types-instructions-and-docs)
- [Telemetry](#telemetry)
- [Distribution and versioning](#distribution-and-versioning)
- [Content migrations](#content-migrations)
- [Instance health](#instance-health)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands-index -->
## Pages
Full records: [pages.md](pages.md)
- [`new`](pages.md#new) - Scaffold a new wiki page of any type.
- [`task new`](pages.md#task-new) - Create one open item in the configured task tracker - never a kb/ page.
- [`task list`](pages.md#task-list) - List a project's open items - id, title, and whether each carries the WAITING status.
- [`task close`](pages.md#task-close) - Mark one tracker item done - never delete it.
- [`touch`](pages.md#touch) - Bump a page's `modified:` date and optionally rewrite its other frontmatter fields.
- [`rename`](pages.md#rename) - Rename a page, or repoint references that name a page that never existed.
- [`rm`](pages.md#rm) - Delete a page and mechanically de-link it from the rest of the wiki.
- [`move`](pages.md#move) - Move a page (or every misplaced page) to the directory its type-spec computes.
## Links and citations
Full records: [links-and-citations.md](links-and-citations.md)
- [`xref add`](links-and-citations.md#xref-add) - Declare that A <rel> B.
- [`xref remove`](links-and-citations.md#xref-remove) - Remove a cross-reference: the inverse of `xref add`.
- [`xref link-source`](links-and-citations.md#xref-link-source) - Batch-link a source page to every entity/concept it mentions.
- [`links show`](links-and-citations.md#links-show) - Show the edges out of and into a page.
- [`cite id`](links-and-citations.md#cite-id) - Print the deterministic footnote id `cite add` would use for this (title, file) pair.
- [`cite add`](links-and-citations.md#cite-add) - Upsert a `[^cite-id]: [[Source - X]]` definition in a page's footnotes region.
- [`cite sync`](links-and-citations.md#cite-sync) - Reconcile each page's footnotes region against its actual `[^id]` references.
## Catalog and log
Full records: [catalog-and-log.md](catalog-and-log.md)
- [`index rebuild`](catalog-and-log.md#index-rebuild) - Regenerate the catalog from every page's frontmatter.
- [`log append`](catalog-and-log.md#log-append) - Append a formatted entry to `kb/log.md`.
- [`log status`](catalog-and-log.md#log-status) - Read-only: count `ingest` entries logged since the last `lint` entry.
## Finding and checking
Full records: [finding-and-checking.md](finding-and-checking.md)
- [`lint`](finding-and-checking.md#lint) - Run structural lint checks against kb/.
- [`search`](finding-and-checking.md#search) - Find pages in `kb/` by text and/or frontmatter.
- [`review`](finding-and-checking.md#review) - The GTD weekly review.
## Provenance
Full records: [provenance.md](provenance.md)
- [`sources coverage`](provenance.md#sources-coverage) - List raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages.
- [`sources trace`](provenance.md#sources-trace) - Trace provenance in either direction: raw file, or page.
- [`sources rebuild-index`](provenance.md#sources-rebuild-index) - Regenerate the `kb/provenance.md` reverse index.
## Raw material and uploads
Full records: [raw-material-and-uploads.md](raw-material-and-uploads.md)
- [`raw fetch`](raw-material-and-uploads.md#raw-fetch) - Capture a web page the user names into `incoming/`: the HTML as received plus a derived text, for `raw accept` to promote.
- [`raw capture`](raw-material-and-uploads.md#raw-capture) - Capture documentation from a git repository into `incoming/<bundle>/`, with a manifest naming the repository, ref rule and commit, for `raw accept` to promote.
- [`raw status`](raw-material-and-uploads.md#raw-status) - Report which captured bundles under `raw/` have fallen behind their repository, with each changed file as `A`/`M`/`D`.
- [`raw pending`](raw-material-and-uploads.md#raw-pending) - List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.
- [`raw accept`](raw-material-and-uploads.md#raw-accept) - Promote one or more files, or one folder, from `incoming/` into `raw/`.
- [`upload list`](raw-material-and-uploads.md#upload-list) - List every MCP submission currently waiting in the quarantine (`mcp-upload/`).
- [`upload show`](raw-material-and-uploads.md#upload-show) - Print one submission's manifest in full.
- [`upload accept`](raw-material-and-uploads.md#upload-accept) - **Upload Review Gate:** promote a submission's file from quarantine into `incoming/`.
- [`upload reject`](raw-material-and-uploads.md#upload-reject) - Delete a submission's material, keeping only its ledger trail.
## Git
Full records: [git.md](git.md)
- [`sync`](git.md#sync) - Fetch `<remote>/<branch>` and bring the local branch up to date with it.
- [`publish`](git.md#publish) - Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
## Guideline export
Full records: [guideline-export.md](guideline-export.md)
- [`export guidelines`](guideline-export.md#export-guidelines) - Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
## Workshop runs and session budget
Full records: [workshop-runs-and-session-budget.md](workshop-runs-and-session-budget.md)
- [`work new`](workshop-runs-and-session-budget.md#work-new) - Scaffold `work/<runkey>/` for one workshop run.
- [`work close`](workshop-runs-and-session-budget.md#work-close) - Delete a finished workshop.
- [`budget status`](workshop-runs-and-session-budget.md#budget-status) - Show the current session's `wikitool` call count and recent command history.
- [`budget reset`](workshop-runs-and-session-budget.md#budget-reset) - Clear the current session's (or every session's) iteration budget state.
## Types, instructions and docs
Full records: [types-instructions-and-docs.md](types-instructions-and-docs.md)
- [`types list`](types-instructions-and-docs.md#types-list) - List every type-spec under `types/`.
- [`types describe`](types-instructions-and-docs.md#types-describe) - Print one type's full contract.
- [`instructions sync`](types-instructions-and-docs.md#instructions-sync) - Publish every `instructions/<name>/SKILL.md` into the harness skill directories.
- [`instructions verify`](types-instructions-and-docs.md#instructions-verify) - Check the instruction layer.
- [`instructions list`](types-instructions-and-docs.md#instructions-list) - List the flat instructions with their descriptions.
- [`docs verify`](types-instructions-and-docs.md#docs-verify) - Check the docs that mirror the code.
- [`docs toc`](types-instructions-and-docs.md#docs-toc) - Create, refresh or remove the generated table-of-contents region.
- [`docs prerequisites`](types-instructions-and-docs.md#docs-prerequisites) - Regenerate `INSTALL.md`'s prerequisites lists from `tools/prerequisites.txt`.
- [`docs contract`](types-instructions-and-docs.md#docs-contract) - Regenerate `tools/CONTRACT.md`'s `<!-- wikitool:commands -->` region.
- [`docs reference`](types-instructions-and-docs.md#docs-reference) - Regenerate the human command reference under `docs/reference/`.
## Telemetry
Full records: [telemetry.md](telemetry.md)
- [`eval sessions`](telemetry.md#eval-sessions) - List the sessions that have a trace under `reports/telemetry/`.
- [`eval score`](telemetry.md#eval-score) - Score one traced session.
## Distribution and versioning
Full records: [distribution-and-versioning.md](distribution-and-versioning.md)
- [`dist export`](distribution-and-versioning.md#dist-export) - Write a contentless, distributable copy of this repo's machinery.
- [`dist adopt`](distribution-and-versioning.md#dist-adopt) - Take shipped templates as this instance's own: copy each to its unsuffixed name.
- [`dist upgrade`](distribution-and-versioning.md#dist-upgrade) - Apply a stack update `dist export` produced - the write half of `version check`.
- [`version show`](distribution-and-versioning.md#version-show) - Print this instance's stack version and where it came from.
- [`version check`](distribution-and-versioning.md#version-check) - Ask the origin's release feed whether a newer stack exists.
- [`version notes`](distribution-and-versioning.md#version-notes) - Print one version's release notes.
- [`version bump`](distribution-and-versioning.md#version-bump) - Raise or continue the one running candidate between two releases.
- [`version regrade`](distribution-and-versioning.md#version-regrade) - List the running candidate's bump titles with their impact grade, or change one or more of them.
- [`version release`](distribution-and-versioning.md#version-release) - Fix the running candidate: strip `VERSION`'s `-beta.N` suffix and close its `CHANGES.md` entry.
## Content migrations
Full records: [content-migrations.md](content-migrations.md)
- [`migrate list`](content-migrations.md#migrate-list) - List every migration document under `instructions/migrations/`.
- [`migrate status`](content-migrations.md#migrate-status) - Show the migrations this instance still owes, in the order they must run.
- [`migrate verify`](content-migrations.md#migrate-verify) - Compare `kb/` against a git revision on the invariants a content migration must not change.
- [`migrate done`](content-migrations.md#migrate-done) - Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json`.
- [`migrate baseline`](content-migrations.md#migrate-baseline) - Declare `kb_version` once, for an instance predating `.wikitool-kb.json`.
## Instance health
Full records: [instance-health.md](instance-health.md)
- [`doctor`](instance-health.md#doctor) - Check that this instance is correctly configured.
<!-- /wikitool:reference-commands-index -->
docs/reference/commands/catalog-and-log.md
+157
View File
@@ -0,0 +1,157 @@
# Catalog and log
The generated catalog of the wiki and its append-only operations log.
<!-- wikitool:toc -->
## Contents
- [`index rebuild`](#index-rebuild)
- [`log append`](#log-append)
- [`log status`](#log-status)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `index rebuild`
Regenerate the catalog from every page's frontmatter.
**Synopsis**
- `wikitool index rebuild [--dry-run]`
**Options**
```text
--dry-run Print what would be written instead of writing it
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool index rebuild
tools/wikitool index rebuild --dry-run
```
**Behaviour**
- Rewrites `kb/index.md` as a map: statistics, one row per collection and per area, and links to the shards - no page rows.
- Writes the page tables to a generated `INDEX.md` in each collection. An area with more than 50 rows gets its own `INDEX.md` in its directory.
- Deletes stale shards - an `INDEX.md` of a collection or area that no longer exists - in the same pass.
- Warns about every page nested more than one directory below its collection; it is still catalogued, folded into its area.
- `--dry-run` prints every file it would write and every stale shard it would remove, and writes nothing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - each catalog file (the map plus one shard per collection/area) is rewritten independently, then stale shards are removed; an interruption can leave some regenerated and others not
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - An I/O error while writing or removing a catalog file (rare)
**See also**
- `wikitool search` - finds a page without reading the catalog
- `wikitool lint` - its Nested Pages finding is what the warning previews
- `instructions/publish-cycle.md` - where a write session runs this
## `log append`
Append a formatted entry to `kb/log.md`.
**Synopsis**
- `wikitool log append --op ingest|query|lint|create|update|delete|rename|move --title "..." [--body "..."|--body-file path]`
**Options**
```text
--op <str> ingest|query|lint|create|update|delete|rename|move
[required]
--title <str> Brief description, e.g. a source path [required]
--body <str> Optional multi-line details
--body-file <path> Read the body from a file instead of --body
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool log append --op ingest --title "raw/articles/docker-cheatsheet.md" --body "Created [[Docker]]; updated [[Container]]."
tools/wikitool log append --op lint --title "2026-09-26" --body-file lint-summary.md
```
**Behaviour**
- Appends one entry to `kb/log.md`: a `## [YYYY-MM-DD] <op> | <title>` heading, the body if one is given, and a `---` separator.
- Not idempotent: every successful run appends a new entry, including a repeated one.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes - single append
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `--op` is not one of ingest, query, lint, create, update, delete, rename, move
- `1` - `--body-file` is missing, not a readable file, or not valid UTF-8
**See also**
- `wikitool log status` - counts the ingests logged since the last lint
- `instructions/publish-cycle.md` - where a write session runs this
## `log status`
Read-only: count `ingest` entries logged since the last `lint` entry.
**Synopsis**
- `wikitool log status`
**Options**
```text
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool log status
```
**Behaviour**
- Counts the `ingest` entries in `kb/log.md` after the most recent `lint` entry, or since the start of the log if it was never linted, and the total number of entries.
- At 10 or more it prints that the `wiki-lint` skill is due next - the every-10-sources full lint of the Maintenance Schedule.
- Read-only; safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `0` - `kb/log.md` is missing or empty - reported as nothing logged, not a failure
**See also**
- `wikitool log append` - writes the entries this counts
- `wiki-lint` skill - what the threshold asks for
- `tools/CONTRACT.md` § Maintenance schedule - the cadence this reports on
<!-- /wikitool:reference-commands -->
docs/reference/commands/content-migrations.md
+267
View File
@@ -0,0 +1,267 @@
# Content migrations
The migrations a release may require of an instance's content, and recording that they ran.
<!-- wikitool:toc -->
## Contents
- [`migrate list`](#migrate-list)
- [`migrate status`](#migrate-status)
- [`migrate verify`](#migrate-verify)
- [`migrate done`](#migrate-done)
- [`migrate baseline`](#migrate-baseline)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `migrate list`
List every migration document under `instructions/migrations/`.
**Synopsis**
- `wikitool migrate list [--json]`
**Options**
```text
--json Print the migrations as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool migrate list
```
**Behaviour**
- Lists every migration document under `instructions/migrations/`, oldest target first, with its kind and obligation.
- Never fails. Read-only and **exempt from the Iteration Budget Gate**.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool migrate status` - which of them this instance still owes
- `instructions/migrate-corpus.md` - how a migration is run
## `migrate status`
Show the migrations this instance still owes, in the order they must run.
**Synopsis**
- `wikitool migrate status [--json]`
**Options**
```text
--json Print the chain as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool migrate status
tools/wikitool migrate status --json
```
**Behaviour**
- Shows every **required** migration whose `migrates_to` lies in `(kb_version, VERSION]`, in the order it must run.
- `offered` migrations are listed separately above the chain: they never block, never count as owed, and are bounded by the applied ledger rather than by `kb_version` - taking one does not move the version.
- With a release stamp present, also reports which shipped files this instance has since edited (from the per-file sha256 in `.wikitool-release.json`) - which says whether an offer may be copied over or has to be reconciled by hand. Without a stamp that question is reported as unanswerable rather than answered.
- Exits 1 only when the content version is undeclared (`.wikitool-kb.json` missing) or `VERSION` is unreadable; it never guesses the content's shape.
- Read-only, safe to retry freely, and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `.wikitool-kb.json` is missing - the content version is undeclared
- `1` - `VERSION` is unreadable
**See also**
- `wikitool migrate done` - records one as applied
- `instructions/migrate-corpus.md` - how a migration is run
- `instructions/upgrade-instance.md` - where an upgrade checks this
## `migrate verify`
Compare `kb/` against a git revision on the invariants a content migration must not change.
**Synopsis**
- `wikitool migrate verify --from <rev> [--path P ...] [--expect-body-change] [--json] [--fail-on-error]`
**Options**
```text
--from <str> Git revision to compare against, e.g. HEAD [required]
--path <str> Limit to a subtree, repeatable (e.g. kb/concepts)
--expect-body-change Also report pages whose body did not change at all
--json Print the diff as JSON
--fail-on-error Exit 1 if any invariant changed
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool migrate verify --from HEAD
tools/wikitool migrate verify --from HEAD --path kb/concepts --expect-body-change
```
**Behaviour**
- Compares `kb/` against the revision `--from` on the invariants a content migration must not change: wikilink and citation **counts** (not sets), footnote definitions, H1, structural frontmatter, and the **count of generated-region marker pairs**.
- Pages are matched by **title**, not path, so a page `move` (or `move --reconcile`) relocated compares as itself - reported separately as `moved` - rather than as a removed-and-added pair.
- Reports added and removed pages without failing on them.
- `--expect-body-change` additionally flags a page whose body did not change at all.
- Not migration-specific: worth running after any bulk rewrite.
- Exits 0 whatever it finds unless `--fail-on-error` is passed.
- Read-only and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Only with `--fail-on-error`: an invariant changed
- `1` - `--from` is not a revision in this repository
**See also**
- `instructions/migrate-corpus.md` - where a migration runs this
- `wikitool lint` - the single-revision checks
## `migrate done`
Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json`.
**Synopsis**
- `wikitool migrate done <version> [--pages N] [--dry-run]`
**Options**
```text
Arguments:
version The migration's target version, e.g. 1.4.0 [required]
Options:
--pages <int> How many pages it touched
--dry-run Report without writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool migrate done 7.0.0 --pages 42 --dry-run
tools/wikitool migrate done 7.0.0 --pages 42
```
**Behaviour**
- Records one migration as applied, advancing `kb_version` in `.wikitool-kb.json` to its target.
- **Refuses any required version that is not the next link in the chain.**
- An `offered` migration is recorded in the applied ledger *without* moving `kb_version` and with no ordering rule applied. Re-recording one already in the ledger is a no-op, not an error - idempotent and safe to repeat.
- Not idempotent for a required migration: it advances the chain.
- `--dry-run` reports without writing.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes - single file write
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unknown version, no `.wikitool-kb.json`, or nothing outstanding
- `1` - A *required* version that is not the next link in the chain
**See also**
- `wikitool migrate status` - the order to apply them in
- `instructions/migrate-corpus.md` - the migration procedure
## `migrate baseline`
Declare `kb_version` once, for an instance predating `.wikitool-kb.json`.
**Synopsis**
- `wikitool migrate baseline <version> [--force]`
**Options**
```text
Arguments:
version The shape this instance's content is already in [required]
Options:
--force Overwrite an existing declaration (not a substitute for `done`)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool migrate baseline 6.2.0
```
**Behaviour**
- Declares `kb_version` once, for an instance predating `.wikitool-kb.json`.
- Refuses to overwrite an existing declaration without `--force`. Advancing the version after a migration is `migrate done`, which checks the chain; this command does not.
- Safe to re-run with the same version.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - single file write
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unparseable version
- `1` - A declaration already exists and `--force` was not passed
**See also**
- `wikitool migrate done` - advances the version after a migration
- `wikitool migrate status` - what is owed from the declared version
<!-- /wikitool:reference-commands -->
docs/reference/commands/distribution-and-versioning.md
+602
View File
@@ -0,0 +1,602 @@
# Distribution and versioning
Exporting, adopting and upgrading an instance, and the version and changelog of the stack itself.
<!-- wikitool:toc -->
## Contents
- [`dist export`](#dist-export)
- [`dist adopt`](#dist-adopt)
- [`dist upgrade`](#dist-upgrade)
- [`version show`](#version-show)
- [`version check`](#version-check)
- [`version notes`](#version-notes)
- [`version bump`](#version-bump)
- [`version regrade`](#version-regrade)
- [`version release`](#version-release)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `dist export`
Write a contentless, distributable copy of this repo's machinery.
**Synopsis**
- `wikitool dist export <target> [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]`
**Options**
```text
Arguments:
target Directory to write the distribution into. Must not exist, or must be
empty. [required]
Options:
--dry-run List what would be written, without writing anything.
--source-repo <str> Repository this export was built from (recorded in the
stamp)
--source-commit <str> Commit this export was built from (recorded in the
stamp)
--release-url <str> Release page this export ships as (recorded in the
stamp)
--update-url <str> Release feed `version check` should ask (recorded in
the stamp)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool dist export ../my-wiki --dry-run
tools/wikitool dist export ../my-wiki
```
**Behaviour**
- Writes a contentless, distributable copy of this repo's machinery into an empty or new `<target>` directory.
- Ships `AGENTS.md`/`README.md`/`EVALS.md` with any `dist:strip-start`...`dist:strip-end` marker region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs, their schemas and their subtype templates `types/<type>.<value>.md` re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim (minus `docs/dev/`), `tools/` (no venv/caches, and not the dev-only test suite `tools/chemenu/tests/` with its `pytest.ini`/`.coveragerc`), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas) and `VERSION`.
- Ships `raw/` and `incoming/` as flat roots, each with a `.gitkeep` and no subdirectories. `incoming/.gitkeep` is trackable and survives becoming a git repository, so a plain clone gets the directory without any bootstrap step.
- Ships templates, never the filled files: `USER.md.template`/`SOUL.md.template`, `kb/CONVENTIONS.md.template`, and each collection's contract re-keyed as `kb/<name>/COLLECTION.md.template`. The filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb/<name>/COLLECTION.md`/`types/<page-type>.md`/`types/<page-type>.<value>.md` bind their instance; `find_leaks` refuses a plan carrying one.
- Writes a generated `.wikitool-release.json` stamp: version, export date, origin, and a sha256 per exported file - the base a later upgrade compares against.
- The four origin options only fill stamp fields: `export` never calls git and cannot discover them.
- A build and test tool: every release is an export packed as a tarball, and an instance is installed from such a release, never from an export directly.
- One-way: no command reconstructs a distributed instance into a dev instance - work on the stack in a clone of the origin repo.
- `--dry-run` lists every file it would write, and writes nothing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - nothing is written until every file is planned
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `<target>` exists and is not empty, or is not a directory
- `1` - The tree has no readable `VERSION`
- `1` - A licence file is missing from the source tree
- `1` - The plan would carry this instance's own data (a filled personalization, conventions or page type-spec file)
**See also**
- `instructions/setup-instance.md` - installs a release, which is this export as a tarball
- `wikitool dist upgrade` - applies a later export to an existing instance
- `wikitool version show` - reads the stamp this writes
## `dist adopt`
Take shipped templates as this instance's own: copy each to its unsuffixed name.
**Synopsis**
- `wikitool dist adopt [<template>...] [--dry-run]`
**Options**
```text
Arguments:
templates... Templates to adopt. Default: every collection contract and page
type-spec template.
Options:
--dry-run List what would be copied, without writing anything.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool dist adopt
tools/wikitool dist adopt types/project.md.template types/project.schema.yaml.template
tools/wikitool dist adopt --dry-run
```
**Behaviour**
- Copies `<name>.template` to `<name>` byte for byte; the template stays where it is, as the base the next `dist upgrade` compares against.
- Without a path: every `kb/<collection>/COLLECTION.md.template` and every `types/*.template` (the `root: kb` page type-specs, their schemas and their subtype templates).
- With paths: exactly those templates, each of which has to be one of the set above.
- Never overwrites: a target that already exists is reported as kept and left untouched.
- Not for `kb/CONVENTIONS.md.template` or the personalization templates - those carry a sentinel and are filled in, not copied.
- `--dry-run` lists what it would copy and keep, and writes nothing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - files are copied one by one; a re-run completes an interrupted one
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - A named path does not exist, or is not a collection contract or page type-spec template
**See also**
- `instructions/setup-instance.md` - adopts every template on a fresh instance
- `instructions/upgrade-instance.md` - adopts a template a release added
- `wikitool dist export` - re-keys these files as `.template` in the first place
## `dist upgrade`
Apply a stack update `dist export` produced - the write half of `version check`.
**Synopsis**
- `wikitool dist upgrade (<source> | --latest [--expect <version>]) [--dry-run] [--keep-local] [--take-release <path>]... [--prune] [--pre]`
**Options**
```text
Arguments:
source An extracted distribution directory, or a release .tar.gz archive - or
use --latest
Options:
--latest Download the release the feed announces as latest
instead of naming a <source>
--expect <str> With --latest: refuse, before downloading anything,
unless the feed's latest release is this version (the
one `version notes` was read for)
--dry-run Classify and report, without writing anything
--keep-local Proceed even with locally changed files - leave each one
untouched rather than aborting
--take-release <str> Overwrite this locally changed path with the release's
version, discarding the local change. Repeatable, and
each path must be one this run reports as locally
changed. The counterpart to --keep-local, which keeps
the change and reports it again on every future upgrade
--prune Accepted and ignored: files the new release no longer
ships are now deleted by default when unchanged since
install
--pre Allow a pre-release (-beta.N) source tree - release.yml
never publishes one
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --dry-run
tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz
tools/wikitool dist upgrade ../chemenu-7.1.0.tar.gz --take-release tools/README.md
tools/wikitool dist upgrade --latest --expect 8.0.0 --dry-run
```
**Behaviour**
- Exactly one of `<source>` and `--latest` names the release. `<source>` is an already-fetched export directory or `.tar.gz` release archive and touches no network: the archive is verified against a sibling `.sha256` if one is present (a missing one is a WARN, not a block) and must unpack to exactly one top-level directory - the shape `.gitea/workflows/release.yml` packs.
- `--latest` asks the release feed (`update_url` of the local stamp, overridable with `$WIKITOOL_UPDATE_URL`; `$WIKITOOL_UPDATE_TOKEN` is sent along) which release is latest, checks that version - `--expect`, a downgrade, a pre-release without `--pre`, already installed - before any download, then downloads the archive and its `.sha256` into a scratch directory removed on every exit. The checksum is mandatory here (a release without one, or an archive that fails it, is an error, not a WARN) and the archive's own `VERSION` must equal the feed's version. The asset URLs are the feed's own `browser_download_url` values, and the token reaches an asset download only if it is on the feed's origin. The checksum protects against transfer errors, not against a feed that is itself compromised - authenticity is the trust in the feed's host.
- `--expect <version>` (only with `--latest`) pins the release the feed may announce: a different latest version is refused before anything is downloaded. Pass the version `wikitool version notes` was read for, in the dry run and in the real run alike.
- The write set is exactly the *new* `.wikitool-release.json`'s `files` block, minus what an export re-seeds from a blank template every time (`kb/log.md`, `raw/.gitkeep`) or seeds once and the instance owns from then on (`.wikitool-kb.json`, `CHANGES.md`), plus the stamp itself, always rewritten.
- Every candidate path is classified against the *local* `.wikitool-release.json`'s recorded digest: unchanged is overwritten silently, absent from the old stamp is created, and locally modified or locally deleted is **never** silently overwritten.
- Locally changed files abort the run with the full list; the abort text names the three answers with the command line filled in, and none of them is the default.
- `--keep-local` proceeds and leaves every locally changed file untouched. The new stamp is still written whole, recording the release's digest for files that were not written, so a skipped file keeps diverging and is reported again on every later run.
- `--take-release <path>` (repeatable) writes the release's version over the named path, discarding the local change, re-creates it if it was locally deleted, and deletes it if the release no longer ships it. The path then matches the stamp and stops being reported.
- The two are decided per path and compose on one call: without `--keep-local`, a locally changed path that no `--take-release` names still aborts the run.
- A path in the old stamp but not the new one is no longer part of the release. Still unchanged since installation, it is deleted, as silently as an unchanged file is overwritten; changed since, it is locally changed like any other and needs one of the same answers - `--take-release` deletes it, `--keep-local` keeps it as the instance's own file, which no later upgrade reports again. Either way it is decided in this run: the new stamp no longer names the path. What the instance owns (`kb/log.md`, `CHANGES.md`, `.wikitool-kb.json`, ...) is never a candidate, and neither is any file the old stamp does not name.
- A directory left holding nothing - at most a `__pycache__/` of `*.pyc` - once those files are deleted is removed with them, up to but never including the instance root. `--prune` is accepted and ignored; it used to switch the deletion on.
- Reports the migration chain the new machinery would owe, but never runs any of it - there is no `migrate run`.
- Reports, but does not block on, a crossed compatibility boundary.
- The local preconditions - `VERSION`, the release stamp, `.wikitool-kb.json`, no outstanding migration, a clean working tree - are checked before the source is read. Not being a git repository at all is a WARN, not a refusal.
- Never touches git - no commit, no push.
- `--dry-run` classifies and reports without writing; a pre-release (`-beta.N`) source needs `--pre`. With `--latest` it still downloads and verifies the archive - that is the only way to classify - and removes it again.
- An interrupted write is not resumed automatically: compare the tree against the printed classification and finish or revert by hand.
- The closing report names `instructions/upgrade-instance.md`, which carries the order for everything after the swap and resumes at `instructions sync`.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes for every refusal - nothing is written. Once writing starts it is a plain sequential file copy with no partial-state cleanup: an interruption mid-copy (killed process, disk full) can leave the tree part-old, part-new
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `0` - The source's version equals the installed one - a no-op success
- `1` - Both `<source>` and `--latest`, or neither; or `--expect` without `--latest`
- `1` - Local `VERSION` missing, or no local `.wikitool-release.json` with a `files` block
- `1` - `.wikitool-kb.json` is missing
- `1` - A migration is already outstanding against the *installed* machinery
- `1` - The working tree is dirty
- `1` - `<source>` does not exist, fails its `.sha256`, or does not unpack to exactly one top-level directory
- `1` - `--latest`: the release feed cannot be reached, or answers with something that is not a release
- `1` - `--latest --expect`: the feed's latest release is another version than the one expected; nothing was downloaded
- `1` - `--latest`: the release publishes no archive or no `.sha256` under the expected name; nothing was downloaded
- `1` - `--latest`: an asset download fails, or the archive fails its `.sha256`
- `1` - `--latest`: the downloaded archive's `VERSION` is not the version the feed announced
- `1` - The source carries no `VERSION`, `.wikitool-release.json` or `files` block
- `1` - The source's version is older than the installed one, or a pre-release without `--pre`
- `1` - A `--take-release` path this run does not classify as locally changed - the one refusal a `--dry-run` also raises
- `1` - Locally changed files that neither `--keep-local` nor a `--take-release` answers for; nothing was written
**See also**
- `wikitool version check` - finds out whether an update exists
- `wikitool version notes` - the notes of the release `--expect` should name
- `instructions/upgrade-instance.md` - the order after the swap
- `INSTALL.md` § "Version und Updates" - which release, whether to take it, where the tarball comes from
- `wikitool migrate status` - the migrations the report names
## `version show`
Print this instance's stack version and where it came from.
**Synopsis**
- `wikitool version show [--json]`
**Options**
```text
--json Print the version and stamp as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version show
tools/wikitool version show --json
```
**Behaviour**
- Prints `VERSION` and where this tree came from: "development tree" without a release stamp, otherwise the export date, source commit and repository from `.wikitool-release.json`, plus the release page when the stamp records one.
- `--json` prints the version, its compatibility key, the stamp and the update URL.
- Bare `wikitool version` is an alias for this.
- Read-only and offline; exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `VERSION` is missing or unparseable
**See also**
- `wikitool version check` - asks whether a newer stack exists
- `wikitool version notes` - prints a version's release notes
## `version check`
Ask the origin's release feed whether a newer stack exists.
**Synopsis**
- `wikitool version check [--url U] [--timeout S] [--json]`
**Options**
```text
--url <str> Release feed to ask (default: the stamp's, else the built-
in origin)
--timeout <float> Seconds to wait for the feed [default: 10.0]
--json Print the result as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version check
tools/wikitool version check --json
```
**Behaviour**
- Asks the release feed for its latest release and compares it with `VERSION`: `state` is `current`, `update`, `migration` (the step crosses a compatibility boundary) or `ahead`.
- The only command whose whole job is the network call - `version notes` reaches the same feed too, but only as a fallback on a distributed instance, and `dist upgrade --latest` asks it which release to download.
- Never reached implicitly from another command (`dist upgrade` asks only when passed `--latest`), needs no key, and times out after `--timeout` seconds (default 10).
- The feed is `--url`, else `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin. `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously.
- For `update` or `migration` it prints that applying the release is a separate, manual step (`INSTALL.md` § "Eine Instanz aktualisieren").
- Read-only; exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only, no local writes
- Counts toward the session call budget: no
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - `VERSION` is missing or unparseable
- `1` - The feed could not be reached, answered non-JSON, or carried no `tag_name`
**See also**
- `wikitool dist upgrade` - applies the update this reports
- `wikitool version notes` - prints the release notes
- `INSTALL.md` § "Version und Updates" - what the operator decides before an update
## `version notes`
Print one version's release notes.
**Synopsis**
- `wikitool version notes [--version X.Y.Z] [--offline] [--url U] [--timeout S]`
**Options**
```text
--version <str> Which entry to print (default: this tree's VERSION)
--offline Never ask the release feed: on a distributed instance,
whose CHANGES.md carries no entry to print, fail with the
release page instead of fetching the notes
--url <str> Release feed to ask for the fallback (default: the stamp's,
as `version check`)
--timeout <float> Seconds to wait for the feed [default: 10.0]
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version notes
tools/wikitool version notes --version 7.0.0
tools/wikitool version notes --offline
```
**Behaviour**
- Prints the `CHANGES.md` entry for `--version` (default: this tree's `VERSION`).
- With no such entry, in a tree with a release stamp (a `dist export` tree) and without `--offline`, it prints the release feed's latest release notes instead. A tree without a stamp (a dev checkout) never asks the feed.
- Only the feed's latest release can be asked for. When that is a different version than requested, stderr names it and the notes are printed anyway - the expected case before an upgrade, where `VERSION` still names the release being left.
- stdout carries nothing but the notes; the line naming the feed being asked and the one naming the release that answered go to stderr. `release.yml` redirects stdout into the file it posts as the release body.
- `--offline` never asks the feed and fails with the stamp's `release_url` instead.
- Every failure names the stamp's `release_url` where it has one, so a run that cannot read the notes is still told where they are.
- Read-only, safe to retry, and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - `--version` is unparseable, `VERSION` is unreadable when `--version` is omitted, or `CHANGES.md` is missing
- `1` - No entry for the requested version in a tree with no release stamp (a dev checkout)
- `1` - No entry for the requested version, and `--offline` was passed
- `1` - No entry for the requested version, and the feed could not be reached or returned a release with an empty `body`
**See also**
- `wikitool version check` - asks the same feed whether a newer stack exists
- `wikitool version bump` - writes the entry this prints
## `version bump`
Raise or continue the one running candidate between two releases.
**Synopsis**
- `wikitool version bump --major|--minor|--patch --title "<...>" [--impact high|medium|low] [--breaking "<what breaks>"] [--no-migration "<reason>"] [--migration-required] [--dry-run]`
**Options**
```text
--major Bump MAJOR (resets MINOR and PATCH)
--minor Bump MINOR (resets PATCH)
--patch Bump PATCH
--title <str> One-line title for the new/updated CHANGES.md entry
[required]
--breaking <str> What stops working (recorded in CHANGES.md). Required on
the bump that first escalates to a boundary crossing,
optional on a later bump of the same crossing candidate
- where it joins the reasons already recorded rather
than replacing them. Refused on a bump that crosses
nothing
--no-migration <str> Why the escalation to a boundary crossing needs no
content migration (recorded in CHANGES.md)
--migration-required Retract this candidate's earlier --no-migration line: a
migration is needed after all. Requires a migration
document already targeting the new base, and refuses
when the entry carries no --no-migration line to
retract.
--impact <str> high|medium|low - how much this bump matters to a reader
of the release notes (default: medium). Grouped into the
entry's bump list; `version regrade` corrects it later
if the running candidate's own judgment changes.
--dry-run Report the change without writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version bump --patch --title "Fix the lint report path" --impact low
tools/wikitool version bump --minor --title "New command: wikitool review" --dry-run
tools/wikitool version bump --major --title "Rename --gate-file to --remotes-file" --breaking "--gate-file is gone; scripts must pass --remotes-file" --no-migration "no content changes"
```
**Behaviour**
- `VERSION` gets a `-beta.N` suffix: one running candidate between two releases, never a fresh number per bump.
- `--major`/`--minor`/`--patch` is **max-wins escalation** against the last release (patch < minor < major): a `--patch` on a MINOR candidate only advances `N`, and escalation never steps back down.
- The first bump of a candidate opens its `CHANGES.md` entry - heading, date, author, and a machine-managed `<!-- wikitool:bumps -->` list of every `--title` collected so far, graded by `--impact` (default `medium`). Every later bump of the same candidate updates that entry in place: one entry per candidate, not one per bump.
- The bump list renders grouped under `**High/Medium/Low impact**` headings, empty groups omitted - except while every bump is `medium`, where it stays one flat list. `version regrade` corrects a grade after the fact.
- Writes the heading, the bump list and the breaking/migration lines; the entry's prose - a summary and one-paragraph `###` sections, one per topic - is left to the author.
- Whenever a required migration document (`migrates_to:` equal to the candidate's base) exists, every bump writes or refreshes `**Migration:** required - <paths>`, the documents' paths sorted and comma-separated, in place of any `none required` line. The two forms never stand side by side.
- Compatibility follows the **leftmost non-zero component** of the candidate's base, which for this stack (at `1.0.0` and up) means MAJOR: PATCH is a fix, MINOR a compatible capability, MAJOR a version that is **not a drop-in replacement** - any hand-work on update, or a downgrade that no longer works. Whether content must be migrated is a second, independent question.
- The bump that first escalates a candidate past the boundary requires `--breaking "<what stops working>"` and, on top of it, a migration document targeting the candidate's base or `--no-migration "<reason>"`. Both are anchored just above the bump list, persist over later bumps of the same candidate without being repeated, and are refused on a bump that crosses nothing at all.
- On a later crossing of the same candidate, a further `--breaking` **joins** the reasons already recorded (flat on the marker line while there is one, bullets under a bare marker from the second on; repeating a reason verbatim is a no-op), while a further `--no-migration` **replaces** the single migration line.
- `--migration-required` replaces the running candidate's `--no-migration` line with the `required` line; it needs a required migration document already targeting the new base. Nothing retracts a recorded `--breaking` reason.
- Enforces that a crossing documents itself, never that the part was chosen correctly.
- Not idempotent: every successful run escalates or continues the candidate again.
- `--dry-run` reports the step without writing.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - `VERSION` then `CHANGES.md`
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Not exactly one of `--major`/`--minor`/`--patch`, an empty `--title`, or an unknown `--impact`
- `1` - `VERSION` or `CHANGES.md` is missing, or `VERSION` and the changelog's newest entry name different versions
- `1` - An escalation to a boundary crossing without `--breaking`, or with neither a migration document targeting the new base nor `--no-migration`
- `1` - `--breaking` or `--no-migration` on a bump that crosses nothing
- `1` - `--no-migration` while a required migration document targets the new base
- `1` - `--migration-required` combined with `--no-migration`, on a bump with no running candidate, with no `--no-migration` line to replace, or without a required migration document already targeting the new base
**See also**
- `wikitool version regrade` - corrects an `--impact` grade
- `wikitool version release` - fixes the candidate into a release
- `instructions/migrate-corpus.md` - how a migration document is written
## `version regrade`
List the running candidate's bump titles with their impact grade, or change one or more of them.
**Synopsis**
- `wikitool version regrade [INDICES...] [--impact high|medium|low]`
**Options**
```text
Arguments:
indices... 1-based positions in the rendered bump list to regrade (see the
bare listing). Omit to just list.
Options:
--impact <str> high|medium|low - required together with indices
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version regrade
tools/wikitool version regrade 3 7 --impact high
```
**Behaviour**
- Without arguments: lists the running candidate's bump titles with their grade, numbered by 1-based position in the rendered list (High before Medium before Low, chronological within a grade). Read-only and exempt from the Iteration Budget Gate.
- With indices and `--impact`: sets the grade of every named position in one call, all resolved against a single read of the current list - `version regrade 3 7 --impact high` grades what is at 3 and 7 now, not 7 after 3 has moved. Writes `CHANGES.md` and is counted by the Iteration Budget Gate.
- The correction path for an `--impact` grade judged at bump time.
- Touches only the topmost entry's bump list - never `VERSION`, never any other part of `CHANGES.md`.
- A write is not idempotent against a changed list: after a first success, the same indices may name different bumps.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - `CHANGES.md` only, and only when indices are given
- Counts toward the session call budget: only when called with arguments
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `VERSION` or `CHANGES.md` is missing, `VERSION` and the changelog's newest entry name different versions, or the topmost entry has no bump list
- `1` - An index outside the rendered list's range, indices given without `--impact`, or an unknown `--impact`
**See also**
- `wikitool version bump` - sets the grade in the first place
- `wikitool version release` - refuses without a summary above this list
## `version release`
Fix the running candidate: strip `VERSION`'s `-beta.N` suffix and close its `CHANGES.md` entry.
**Synopsis**
- `wikitool version release [--title "<...>"] [--dry-run]`
**Options**
```text
--title <str> Replace the entry's heading title (default: the last bump's)
--dry-run Report the change without writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool version release --dry-run
tools/wikitool version release --title "Command records rewritten for agents"
```
**Behaviour**
- Strips `VERSION`'s `-beta.N` suffix - the candidate's base becomes the release - and closes the candidate's `CHANGES.md` entry.
- Without `--title` the heading keeps whichever bump last set it; `--title` replaces it - the normal case for a candidate that collected several bumps, whose entry wants a summarising heading rather than the most recent one.
- Leaves the entry's machine-managed bump list untouched, as the record of what happened.
- From two bumps on, requires a summary paragraph (at least 200 non-whitespace characters) between the bump list and the first `###` section heading. A candidate with exactly one bump is exempt, since there its own paragraph already is the summary. `--dry-run` runs this check too.
- Refuses an entry that would exceed its budget once released: at most 32 000 bytes (UTF-8) as `version notes` prints it, and every `###` section exactly one paragraph of at most 1 000 bytes, heading line excluded. The same check as `docs verify`, run on the released heading; `--dry-run` runs it too.
- Commits nothing and pushes nothing. The following `publish` moves `VERSION` onto `main`, which `release.yml` reacts to.
- Not idempotent: a second run fails once the suffix is gone.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - `VERSION` then `CHANGES.md`
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `VERSION` is already a release - there is no running candidate to fix
- `1` - `VERSION` or `CHANGES.md` is missing, or `VERSION` and the changelog's newest entry name different versions
- `1` - Two or more bumps and no summary paragraph above the `###` sections
- `1` - The entry is over 32 000 bytes, or a `###` section has more than one paragraph or a paragraph over 1 000 bytes - the error names each with its size
**See also**
- `wikitool version bump` - opens and continues the candidate
- `wikitool version regrade` - shows the bump list the summary sits under
- `wikitool publish` - moves the released `VERSION` onto `main`
<!-- /wikitool:reference-commands -->
docs/reference/commands/finding-and-checking.md
+214
View File
@@ -0,0 +1,214 @@
# Finding and checking
Searching the wiki, checking its health, and preparing the weekly review of projects and commitments.
<!-- wikitool:toc -->
## Contents
- [`lint`](#lint)
- [`search`](#search)
- [`review`](#review)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `lint`
Run structural lint checks against kb/.
**Synopsis**
- `wikitool lint [--json] [--markdown out.md] [--full] [--fail-on-error]`
**Options**
```text
--json Print the raw findings as JSON and write no report
--markdown <path> Write the markdown report here instead of the default
reports/Lint Report <date>.md
--full Print the whole report instead of only the sections with
findings
--fail-on-error Exit non-zero if hard errors were found
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool lint
tools/wikitool lint --json
tools/wikitool lint --fail-on-error
```
**Behaviour**
- Structural and provenance checks over `kb/`: broken wikilinks, wikilinks wrapped across a line break, dangling frontmatter references, orphan pages, index drift, schema gaps, duplicate titles, title mismatches, uncovered raw files, broken `raw_files:` refs, raw files claimed by more than one source page, unmarked provenance, citation/frontmatter drift, and unbalanced generated-region markers.
- Unportable Titles is a hard finding, and hard at every `kb_version`: a page whose title is not a valid file name on Windows and macOS (forbidden character, reserved name, trailing dot or space), or that collides with another page by case or Unicode normalization. `wikitool rename` is the fix.
- Wrapped Wikilinks is a hard finding: a `[[...]]` with a line break inside it. The link graph reads it as the title it folds to (the break and its indentation become one space), so it is not also a broken link unless that title is missing; the fix is to put it back on one line.
- Broken Anchors is advisory: a `[[Page#Section]]` whose page exists but has no heading the anchor names (any level, compared without case, inline-code backticks or extra whitespace; every segment of a nested `[[Page#A#B]]`). The link still reaches the page, so nothing else reports it - typically a section that was promoted to a page of its own or renamed. A missing page is Broken Wikilinks instead.
- Pages nested more than one directory below their collection are a hard finding - the generated catalog folds these into their area silently rather than merely reading it.
- Edges whose label is missing or not authorised by the source collection's `outbound:` are both hard once `kb_version` has reached the release that introduced labelled edges, and advisory below it.
- Advisory only: `see-also` edges whose reverse direction already carries a specific label - never migration-gated.
- Advisory only: a collection past the catalog's per-area shard threshold that has no areas to shard, reported with the split its subtype field would produce, and only when that split puts every resulting area at or under the threshold.
- Advisory only: source pages sitting in the `unclassified` catalog slot.
- Advisory only: Long Paths - a file under `kb/` or `raw/` whose path below the instance root is over 160 UTF-16 code units, the budget that keeps a Windows checkout without long paths working. Reported as `{path, length}`; a corpus over the budget breaks no lint run. `wikitool rename` is the fix for a page.
- Advisory only: quote-limit overages (>2 blockquotes/page).
- Advisory only: Unfilled Template Sections - a page with at least one `##` section whose non-blank lines are all template placeholders (content starting with `TODO`, after an optional list marker, checkbox, table cell or bold field label), reported as `{page, sections}`. Code, generated regions and footnote definitions do not count; an empty section, or one placeholder beside written lines, is no finding. Writing the section from the page's sources or retiring the page is the fix - not a mechanical one.
- Prints only the sections that found something and always writes the full report to `reports/Lint Report <date>.md` (or `--markdown`), naming the path. `--full` prints everything; `--json` prints the findings and writes nothing.
- Exits 0 whatever it finds unless `--fail-on-error` is passed.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Writes one report file (single atomic write) unless `--json`
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Only with `--fail-on-error`: hard findings exist
**See also**
- `wiki-lint` skill - the procedure that runs this
- `wikitool move --reconcile` - fixes Misplaced and Nested Pages
- `wikitool rename` - fixes Unportable Titles and, for a page, Long Paths
- `wikitool log status` - whether a full lint is due
## `search`
Find pages in `kb/` by text and/or frontmatter.
**Synopsis**
- `wikitool search ["<text>"] [--field <predicate> ...] [--kind/--subtype/--collection/--tag <v>] [--regex] [--limit N] [--sort [-]<field>] [--backend <name>] [--matches] [--json]`
**Options**
```text
Arguments:
text Text to search for. Omit it to run a pure frontmatter query.
Options:
-f, --field <str> Frontmatter predicate, repeatable (AND). Forms:
field=value, field~substring, 'field>=value', 'field:*'
(present), '!field' (absent).
--kind <str> Shorthand for --field kind=<value>.
--subtype <str> Shorthand for --field subtype=<value>.
--collection <str> Shorthand for --field collection=<value>.
--tag <str> Shorthand for --field tags=<value>.
--regex Treat the query as a regex. Off by default: terms are
literal.
--limit <int> Maximum number of results. 0 for no limit. A capped result
says so. [default: 50]
--sort <str> Sort by a result field; prefix with '-' to reverse, e.g.
-modified.
--backend <str> Search backend(s), comma-separated. Default 'rg' (or
$WIKITOOL_SEARCH_BACKEND).
--matches Print the matching lines under each result.
--json Print the results as JSON.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool search "act runner"
tools/wikitool search --field entity_type=system --field '!sources'
tools/wikitool search "docker" --collection entities --limit 10 --json
```
**Behaviour**
- Finds pages in `kb/` without reading the catalog. Text search runs through a pluggable backend (`rg` today).
- `--field` predicates are evaluated on frontmatter - `f=v`, `f~substring`, `'f>=v'`, `'f:*'` (present), `'!f'` (absent) - repeatable and ANDed. With no text this is a pure structured query.
- One hit per line, ` | `-separated as `score | kind/subtype | title | path | summary`. **Title and path are never truncated** - the title is the identifier `touch`/`xref`/`cite` take. The summary, the one lossy field and the only one that may contain the separator, goes last, so splitting on `" | "` with `maxsplit=4` is unambiguous.
- Scope is pages: the backend walks `kb/` but drops the kb-root meta files, every `COLLECTION.md` and every generated `INDEX.md` - a hand-run grep over `kb/` can add none of them but those.
- `--limit` defaults to 50 (`0` for no limit), and **a truncated result says so**: `50 of 182 result(s)` in the table, `total`/`truncated`/`limit` beside `count` in `--json`, where `count` stays the number of results in the payload. `api.search` and the MCP `search` tool carry the same default and the same fields.
- A page whose frontmatter does not parse can match no positive predicate, so it is **named** rather than dropped: `--json` always carries an `unreadable` list of `{path, reason}` (usually empty), and the table form writes the same lines to stderr.
- An unknown field name is reported with the list of fields that do exist - never answered with an empty result, which would read as "no such pages".
- `--regex` is applied by `rg` alone, whose engine is linear; the ranking boosts for title and summary are literal-containment only, so a non-literal pattern is ranked by match count.
- `rg` is killed after 30 s and reported as a failure.
- Read-only, and **exempt from the Iteration Budget Gate**.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `rg` is not installed
- `1` - `rg` did not finish within 30 s
- `1` - A malformed `--field` predicate, or an unknown `--backend`
- `1` - An unknown field name; the error lists the fields that exist
**See also**
- `wikitool links show` - the edges around a page once found
- `wiki-query` skill - answering a question from the wiki
## `review`
The GTD weekly review.
**Synopsis**
- `wikitool review [--json]`
**Options**
```text
--json Print the findings as JSON.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool review
tools/wikitool review --json
```
**Behaviour**
- Joins the configured task-tracker provider against `kb/gtd/` project pages over the case-normalized project name, at read time, storing nothing - not even a `reports/` file.
- **stalled**: a tracker project with zero open items whose `kb/` page is `state: active`; `dormant`/`completed`/`abandoned` never fire.
- **waiting-overdue**: a `WAITING` item whose `follow_up_at` is older than `thresholds.stalled_waiting_days`.
- **unpaged-project**: a tracker project with no matching `kb/` page, older than `thresholds.unpaged_project_weeks`.
- **no-open-loop**: a `kb/` page `state: active` with no matching tracker project, or one with zero open items - the reverse direction of unpaged-project, so a rename on either side surfaces on both.
- **someday-stale**: a someday/maybe item untouched for longer than `thresholds.someday_stale_months`.
- A value a provider genuinely cannot supply - a `WAITING` item with no `follow_up_at`, a tracker project with no determinable creation date - is its own finding (`waiting_no_follow_up`/`project_age_unknown`) rather than a silent skip.
- Thresholds come from the tracker configuration, never from the schema.
- The tracker configuration is read from `.wikitool-tasks.json`, or from the file `WIKITOOL_TASKS_CONFIG` names when that variable is set (so one checkout can be run against several trackers in turn). A set variable that names no file is an error, never "no tracker configured".
- Text output is one `[check] project: message` line per finding, preceded by a `Source:` line naming which access path answered and, for `superproductivity`'s `access: "snapshot"`, the snapshot's age. `--json` carries the same findings plus `checks_run`/`checks_skipped`/`kb_project_count`/`complete`/`source` (`{"kind": ..., "detail": ...}` or `null`).
- A provider that cannot be reached mid-run degrades only the checks that needed the failing call; the report is printed in full, then exit 1 follows - never rendered as if it were complete.
- Re-reads everything fresh on every call, so nothing is ever stale to re-fetch.
- Read-only, and **exempt from the Iteration Budget Gate**.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - No `.wikitool-tasks.json`, or a malformed one - a clear "no tracker configured" message
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
- `1` - The provider was reachable at config-parse time but a read call failed mid-run; the full report (findings plus which checks ran) was printed first
**See also**
- `gtd-weekly-review` skill - turns the findings into decisions
- `wikitool task list` / `wikitool task close` - act on an item
- `wikitool new project` - a project page and its tracker project
<!-- /wikitool:reference-commands -->
docs/reference/commands/git.md
+160
View File
@@ -0,0 +1,160 @@
# Git
Pulling in the remote and publishing a change - the only way a change leaves this checkout.
<!-- wikitool:toc -->
## Contents
- [`sync`](#sync)
- [`publish`](#publish)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `sync`
Fetch `<remote>/<branch>` and bring the local branch up to date with it.
**Synopsis**
- `wikitool sync [--remote origin] [--branch main] [--confirm-rebase TOKEN]`
**Options**
```text
--remote <str> Git remote to reconcile against [default: origin]
--branch <str> Branch to reconcile [default: main]
--confirm-rebase TOKEN Clear the rebase-review gate for the exact upstream
state and file overlap this token was issued for. The
token comes from the gate's own refusal output.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool sync
tools/wikitool sync --confirm-rebase <token> # re-run after exit 42, once the user approved
```
**Behaviour**
- Fetches `<remote>/<branch>`, then: fast-forwards when only the remote moved; rebases the local commits on top when both sides moved but touched disjoint files; exits 42 (rebase-review gate) when both sides touched the same file.
- Files `wikitool` generates (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `kb/**/INDEX.md`) never count as an overlap and never reach the gate. Where they are what stops git - uncommitted ones in the way of a fast-forward or rebase, or a rebase conflict in them - they are merged mechanically: `kb/log.md` becomes the incoming log plus the entries only this side appended, and the catalog and `kb/provenance.md` are regenerated from the merged pages and left as an uncommitted change. A reconcile git can do on its own is unchanged and regenerates nothing.
- Uncommitted changes that no incoming commit touches do not stop a rebase: they are carried through it (`git rebase --autostash`) and come back unchanged.
- Before anything is set aside, the working tree is stored as a commit under `refs/wikitool/reconcile-backup`. A successful reconcile drops it; a failed one restores the working tree and names the ref.
- A refused call performs no rebase attempt and leaves the branch where it was.
- The `--confirm-rebase` token covers the exact upstream state and the set of files touched on both sides, generated files excluded; either one moving makes it stale.
- Makes no commit, no push, and no forced operation of any kind.
- Three messages for a fetch that fails: no remote of that name, a remote that answers but has no such branch yet (a new, empty repository), and a remote that cannot be reached. All three exit 0 here; `publish` stops on the first and the last.
- Run it once at the start of a writing session.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - fetch, then at most one merge/rebase attempt, aborted cleanly on failure with the working tree restored; a success that merged generated files then rewrites the catalog
- Counts toward the session call budget: yes
- Can reach the network: yes
- Gates: rebase-review - see [exit codes and gates](../exit-codes.md)
**Exit codes**
- `0` - success
- `0` - No remote configured - reported and skipped, not a failure
- `0` - The remote cannot be reached - reported and skipped, not a failure
- `1` - The automatic rebase hit a real conflict outside the generated files (git failed), or `kb/log.md` was edited rather than appended to; it is aborted cleanly and the working tree restored
- `1` - An uncommitted change - or an untracked file - sits on a path the incoming commits also change; nothing was changed
- `42` - Rebase-review gate: `<remote>/<branch>` moved and both sides changed the same file, generated files aside; the output lists the upstream commits, the overlapping files and their diff
**See also**
- `wikitool publish` - runs the same reconcile before it commits and pushes
- `instructions/session-setup.md` - where a session runs `sync`
- `instructions/gates.md` - the gate procedure
## `publish`
Reconcile with `<remote>/<branch>`, then stage all changes, commit, and push.
**Synopsis**
- `wikitool publish --message "<op>: <desc>" [--no-push] [--confirm TOKEN] [--confirm-rebase TOKEN] [--threshold N] [--remote origin] [--branch main] [--path P ...]`
**Options**
```text
--message <str> Commit message summary, e.g. 'ingest: docker-
cheatsheet' [required]
--push / --no-push [default: push]
--confirm TOKEN Clear the Mass-Update Gate for the exact changeset
this token was issued for. The token comes from the
gate's own refusal output - run without it first to
see the file list, show that output to the user, and
only pass this once they have approved it.
--confirm-rebase TOKEN Clear the rebase-review gate raised by the pre-push
reconcile step. The token comes from that gate's own
refusal output - see `tools/wikitool sync`.
--threshold <int> Mass-Update Gate threshold: number of counted files
at/above which clearance is required. [default: 10]
--remote <str> Git remote to push to [default: origin]
--branch <str> Branch to push [default: main]
--path <str> Limit this publish to a path (repeatable). Lets a
large change be committed in reviewable batches
instead of one opaque commit - the Mass-Update Gate
then counts only the files in scope.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool publish --message "ingest: docker-cheatsheet"
tools/wikitool publish --confirm <token> --message "ingest: docker-cheatsheet" # re-run after a Mass-Update exit 42, once the user approved
tools/wikitool publish --confirm-rebase <token> --message "ingest: docker-cheatsheet" # re-run after a rebase-review exit 42, once the user approved
```
**Behaviour**
- Order: branch check and Publish-Remote Gate, then the reconcile with `<remote>/<branch>`, then the Mass-Update Gate, then `git add -A`, commit and push. `--no-push` skips all but the Mass-Update Gate and the commit.
- Without `--no-push`, a remote that is not configured or cannot be reached ends the call with exit 1 at the reconcile - before the gate, `git add` and the commit, and also on a clean tree. Nothing is committed, the index and the working tree are unchanged, and the message names `--no-push` as the way to a local commit. The next `publish` that reaches the remote pushes that commit along with whatever is new. A remote that answers but has no `<branch>` yet (a new, empty repository) is not this case: the first publish of an instance commits and pushes as before.
- Reconcile: fetches `<remote>/<branch>`, fast-forwards when only the remote moved, rebases the local commits on top when both sides moved but touched disjoint files, and exits 42 (rebase-review gate) when both sides touched the same file. A refused reconcile performs no rebase attempt. The `--confirm-rebase` token covers the exact upstream state and the set of files touched on both sides.
- Generated files are never an overlap: where they are all that stops the reconcile, it merges them as `sync` does (both sides' log entries kept, the catalog and `kb/provenance.md` regenerated) and carries uncommitted changes no incoming commit touches through the rebase. The proactive reconcile runs before staging, so this publish commits the regenerated files - with `--path` too, see there; on the retry after a rejected push they get a commit of their own before the second push.
- The push target must be the checked-out branch; this is checked before anything is staged. The unborn branch of a fresh `git init -b main` counts as checked out, so the first publish of a new instance works; a real detached HEAD is refused.
- With nothing new to stage, a local commit the remote lacks is still pushed: one left behind by an earlier publish whose push failed, or every commit when the remote answers but does not have the branch yet (a new, empty remote repository).
- A rejected push that finds the remote unreachable on its one retry reports the original push error.
- A rejected push gets exactly one more reconcile-and-push; never more than one.
- Mass-Update Gate: counts the files that would be committed, refuses with exit 42 at `--threshold` (default 10) or more, and prints a review report - a scale line (file count, total lines added/removed, status breakdown), attention notes where they apply (deletions by name, control-plane and harness-config touches, published pages, the largest single change, binaries), and every counted path grouped by area with its status and churn. The list is what the commit will hold: it is computed from a scratch copy of the index after `git add -A`, so a path that is staged as deleted and back in the working tree is not counted twice, and a rename counts as its old path deleted plus its new path added. The real index and the working tree are not touched, so a refused publish leaves both byte-identical.
- Never counted and never shown for approval, but committed like everything else: anything under `work/`, and the files `wikitool` generates itself (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`). The refusal line accounts for both, by reason.
- The `--confirm` token covers each counted path, the blob id of its contents and the publish target: a different file list or edited contents need a new clearance.
- Publish-Remote Gate: when the checkout carries `.wikitool-remotes.json` and the push URL of `--remote` is not listed in it, exits 42 before the reconcile fetches anything. The URL is read with `git remote get-url --push`, so a repointed remote does not pass on its name. An absent file means unrestricted; a malformed one is an error, not permission.
- `--path` (repeatable) scopes the whole operation - gate count, staging and commit - to that subtree, with one exception: uncommitted generated files outside `--path` (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md` under `kb/`) are staged and committed too, and a line names them - whether a reconcile or the change itself wrote them. They stay out of the gate count, and the `--confirm` token and the re-run line cover the `--path` given. Every other file outside `--path` is left as it is.
- Commit message: `--message`, then a `Files changed:` paragraph listing every committed path. When `--message` ends in a paragraph git reads as a trailer block (`Co-Authored-By:` and the like), that block stays the last paragraph and the list goes in front of it, since git reads trailers from the last paragraph only. `git interpret-trailers` decides whether there is such a block, and the list moves only when git reads the same trailers from the result; otherwise it is appended at the end. `--message` is not part of any gate token.
- After a successful commit or push whose changed files include `tools/`, `types/`, `instructions/`, `AGENTS.md` or a path ending in `CONTRACT.md`, prints one reminder line: the phase past this point (an issue-body rewrite, `docs/` staleness, a changelog entry's accuracy) is not covered by `docs verify`, `instructions verify` or `pytest`. It is not a gate: no exit code change, nothing to clear, and silent for an ordinary content publish.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - sequential git operations. Every gate runs before staging, except on the one retry of a rejected push, where the rebase-review gate can exit 42 after the commit: nothing is pushed, and the re-run with `--confirm-rebase` pushes that commit
- Counts toward the session call budget: yes
- Can reach the network: yes
- Gates: mass-update, publish-remote, rebase-review - see [exit codes and gates](../exit-codes.md)
**Exit codes**
- `0` - success
- `1` - git failed - `git add`, `git commit`, `git push`, or the reconcile's automatic rebase (a conflict outside the generated files, an edited `kb/log.md`, or an uncommitted change on a path the incoming commits change)
- `1` - The push target (`--branch`) is not the checked-out branch, or HEAD is detached; the unborn branch of a fresh `git init` is not this case
- `1` - No `--no-push`, and the remote is not configured or cannot be reached; nothing was committed
- `1` - `--yes`/`-y` was passed - the flag does not exist and fails with an explicit error
- `1` - `.wikitool-remotes.json` is unreadable or has no usable `allowed_push_urls` list
- `42` - Mass-Update Gate: `--threshold` (default 10) or more counted files would be committed, or the `--confirm` token does not match this changeset
- `42` - Rebase-review gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff
- `42` - Publish-Remote Gate: `.wikitool-remotes.json` exists and the push URL of `--remote` is not listed in it, or `--remote` resolves to no push URL
**See also**
- `wikitool sync` - the same reconcile on its own, without committing or pushing
- `instructions/gates.md` - the gate procedure
- `instructions/setup-instance.md` - the first publish of a new instance
<!-- /wikitool:reference-commands -->
docs/reference/commands/guideline-export.md
+77
View File
@@ -0,0 +1,77 @@
# Guideline export
Writing the generated `GUIDELINES.md` into the repositories the wiki has captured.
<!-- wikitool:reference-commands -->
## `export guidelines`
Render the guideline pages into one generated `GUIDELINES.md`, or push it into the captured repositories that opted in (**Guideline Push Gate**).
**Synopsis**
- `wikitool export guidelines [--field <predicate> ...] [--kind/--subtype/--collection/--tag <v>]` - Prints the file to stdout; no network.
- `wikitool export guidelines <predicates> --push [--bundle <raw-bundle> ...] [--confirm TOKEN]` - Writes it into the captured repositories, behind the gate.
**Options**
```text
-f, --field <str> Frontmatter predicate, repeatable (AND) - the same forms
as `search --field`.
--kind <str> Shorthand for --field kind=<value>.
--subtype <str> Shorthand for --field subtype=<value>.
--collection <str> Shorthand for --field collection=<value>.
--tag <str> Shorthand for --field tags=<value>.
--push Write the file into the captured repositories (Guideline
Push Gate).
--bundle <path> Only the repository of this captured bundle under raw/;
repeatable.
--confirm <str> The token from a prior exit 42, once a human has approved
it.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool export guidelines --tag guideline
tools/wikitool export guidelines --tag guideline --push
tools/wikitool export guidelines --confirm <token> --push --field tags=guideline # re-run after exit 42, once the user approved every diff
```
**Behaviour**
- Selects pages with `search`'s own predicates and no text: `--field` (repeatable, AND), `--kind`, `--subtype`, `--collection`, `--tag`. Which filter selects this instance's guidelines is written down in `kb/CONVENTIONS.md`.
- Renders mechanically: line 1 is `<!-- wikitool:export kind=guidelines instance=<origin URL without userinfo, or local> commit=<newest commit touching a selected page> - generated, do not edit by hand -->`; then each page sorted by title, as `# <title>` and its text without frontmatter, the generated links and footnote blocks, citation markers (`[^id]`, legacy `^[[...]]`), with `[[Title|Text]]` as `Text` and `[[Title]]` as `Title`. A leading H1 of the page is replaced by the title heading. Code is never rewritten. LF endings, no timestamp: the same pages give the same bytes.
- Refuses before printing or pushing anything: no predicate, no match, an unknown field, any page under `kb/` with unreadable frontmatter, uncommitted changes in `kb/` or `types/` (untracked files included), or no git checkout.
- `--push` targets every captured repository - each `(repo, ref)` from a `_capture.json` under `raw/`, `--bundle` narrowing it. A target is skipped with its reason when its ref rule is a tag pattern, it cannot be reached or has no such branch, its root has no `GUIDELINES.md` (not opted in), its `GUIDELINES.md` has no export header (hand-written), or the header names another `instance=`. A header line alone is the opt-in.
- Guideline Push Gate: `--push` without a matching `--confirm` fetches and builds everything, pushes nothing, and exits 42 with each target's status (`new`, `changed`, `unchanged`, `skipped`), the diff for every target it would write, and the re-run line. The token digests URL, branch, old tip and new blob of every target to write.
- With a matching token, each target gets exactly one commit on the fetched tip changing only `GUIDELINES.md`, authored as the instance checkout's `user.name`/`user.email`, pushed without force; one line per target: `unchanged`, `written <old> -> <new>`, `skipped (<reason>)`, `rejected (<reason>)` or `failed (<reason>)`.
- Nothing to write (every target unchanged or skipped) ends with exit 0 and no gate. A second run after a successful one finds every target unchanged.
- The target repository's own `AGENTS.md` points at the file (Claude Code: `@GUIDELINES.md`); that belongs to the repository, not to this command.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Per target - one commit pushed without force, or nothing; targets already written stay written when a later one fails
- Counts toward the session call budget: yes
- Can reach the network: yes
- Gates: guideline-push - see [exit codes and gates](../exit-codes.md)
**Exit codes**
- `0` - success
- `0` - A target was skipped - tag rule, not reachable, not opted in, hand-written file, another instance's file
- `1` - No predicate, no page matches, a malformed predicate or an unknown field
- `1` - A page under `kb/` has unreadable frontmatter, `kb/` or `types/` has uncommitted changes, or the instance is not a git checkout
- `1` - `--push` with no `user.name` or `user.email` in the instance checkout's git configuration - checked before any target is fetched
- `1` - `--bundle` names no captured bundle under `raw/`, or `--bundle`/`--confirm` was given without `--push`
- `1` - A target rejected the push (its branch moved since the fetch) or a git call failed - the other targets were still served
- `42` - Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets
**See also**
- `wikitool search` - the same predicates, to preview the selection
- `wikitool raw capture` - how a repository becomes a target
- `instructions/gates.md` - the gate procedure
<!-- /wikitool:reference-commands -->
docs/reference/commands/instance-health.md
+66
View File
@@ -0,0 +1,66 @@
# Instance health
A single check of whether this instance is set up and working.
<!-- wikitool:reference-commands -->
## `doctor`
Check that this instance is correctly configured.
**Synopsis**
- `wikitool doctor [--json]`
**Options**
```text
--json Print the checks as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool doctor
tools/wikitool doctor --json
```
**Behaviour**
- Checks dependencies (Python, ripgrep), author resolution, stack version, git identity/branch/remote, published skills, the kb/raw/reports/work/instructions structure, and generated files.
- Tool paths (`tool-paths`): `.wikitool-tools.json` written by a preflight that finished, every tool `tools/prerequisites.txt` names for this platform recorded, and every recorded path still there - a `FAIL` otherwise, fixed by running `tools/preflight.sh` again (PowerShell 7: `tools/preflight.ps1`).
- Install folder (`install-dir`): on Windows with long paths off, a `FAIL` when the folder holding `tools/` is longer than `tools/prerequisites.txt` allows (95 characters) - the limit the preflight enforces before it sets anything up.
- PowerShell (`execution-policy`, `script-marks`; Windows only, `OK` elsewhere): a `FAIL` when the effective execution policy is `Restricted` or `AllSigned` - the line then says whether a group policy sets it, which only whoever administers the computer can change - and a `FAIL` when a script under `tools/` carries a Mark of the Web from the internet zone, which a browser download unpacked in Explorer leaves behind and `Invoke-WebRequest` plus `tar` do not. `tools/preflight.ps1` checks the same two things before it stops.
- Personalization: `USER.md`/`SOUL.md` present **and** filled - a file still carrying the template's sentinel is a `FAIL`.
- KB conventions: `kb/CONVENTIONS.md` present, unsentinelled, and naming all three tool-owned section headings - a `FAIL` on any of the three.
- Environment note: `ENVIRONMENT.md` is optional, so absent is `OK`; a still-templated one is a `WARN`.
- MCP `submit` tool: whether `.wikitool-upload.json` is present, absent or malformed, its limits, and how many submissions wait in `mcp-upload/`. Absent is `OK` and means the write path does not exist at all; malformed is a `FAIL`.
- Task tracker: `.wikitool-tasks.json` present, absent or malformed - absent is `OK` (no tracker configured), malformed is a `FAIL`. When `WIKITOOL_TASKS_CONFIG` is set, that file is read instead and the finding names it; a set variable that names no file is a `FAIL`, never `OK`.
- For a configured `superproductivity` provider, the configured `access` path's own state: `access: "api"` reports whether its local REST API answers `GET /health` with a ready renderer right now, `access: "snapshot"` whether a backup file is ready. The other access path is never attempted, and neither state is ever a `FAIL`.
- For a configured `caldav` provider, whether the server is reachable and Basic auth succeeds - never a `FAIL`; only a broken config block is.
- Session id source: `OK` for `WIKITOOL_SESSION_ID` or a registered harness variable, `WARN` only for the bare parent-pid fallback.
- Telemetry: on or off and why - installation-form default, `.wikitool-telemetry.json`, or `WIKI_TRACE` - and the current session's count and byte total against both caps; never a `FAIL`.
- Exits 1 only on a `FAIL`; a missing remote, session id or `VERSION` is a `WARN`, not a fault.
- Read-only and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - At least one check reported `FAIL` (a `WARN`, e.g. no remote or no `WIKITOOL_SESSION_ID`, does not exit 1)
**See also**
- `instructions/setup-instance.md` - the setup steps most findings point back to
- `instructions/preflight.md` - what `tool-paths` and `install-dir` point back to
- `INSTALL.md` § "Konfiguration" - the per-checkout configuration files
- `EVALS.md` - telemetry state and caps
- `instructions/session-setup.md` - setting `WIKITOOL_SESSION_ID`
<!-- /wikitool:reference-commands -->
docs/reference/commands/links-and-citations.md
+376
View File
@@ -0,0 +1,376 @@
# Links and citations
Declaring the relationships between pages and the citations that tie a claim to its source, with the generated regions that hold both.
<!-- wikitool:toc -->
## Contents
- [`xref add`](#xref-add)
- [`xref remove`](#xref-remove)
- [`xref link-source`](#xref-link-source)
- [`links show`](#links-show)
- [`cite id`](#cite-id)
- [`cite add`](#cite-add)
- [`cite sync`](#cite-sync)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `xref add`
Declare that A <rel> B.
**Synopsis**
- `wikitool xref add --a "<A>" --b "<B>" --rel <label> [--dry-run]`
**Options**
```text
--a <str> Exact title of the page that asserts the edge [required]
--b <str> Exact title of the page it points at [required]
--rel <str> Label from instructions/link-taxonomy.md, e.g. depends-on
[required]
--dry-run Preview the change instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool xref add --a "Gitea Actions" --b "Act Runner" --rel depends-on
tools/wikitool xref add --a "Gitea Actions" --b "Act Runner" --rel depends-on --dry-run
```
**Behaviour**
- Declares **one** edge, `A <label> B`: written into A's `related:` as `- <label>: B` and rendered into A's generated links region.
- B is not touched and does not point back - its inbound view is rendered from the graph.
- Idempotent; re-running with a different label *relabels* rather than appending, since one page asserts one thing about another.
- Refuses before writing when A's type does not declare `related:` - a source page declares `entities:`/`concepts:` instead, and the refusal names them and points at `xref link-source`.
- Refuses before writing when `<label>` is not authorised by the source collection's `outbound:` block for the target's collection; the refusal lists the authorised set and points at `instructions/link-taxonomy.md`.
- `--dry-run` reports the edge without writing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - a single write to A; B is never touched, and every refusal happens before it
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Page A or B not found
- `1` - A's type declares no `related:` field
- `1` - `<label>` is not authorised for this collection pair, or the source collection authorises no labels into the target's collection at all
**See also**
- `wikitool xref remove` - removes a reference
- `wikitool links show` - the edges out of and into a page
- `instructions/link-taxonomy.md` - the labels and what each asserts
## `xref remove`
Remove a cross-reference: the inverse of `xref add`.
**Synopsis**
- `wikitool xref remove --a "<A>" --b "<B>" [--dry-run]`
**Options**
```text
--a <str> Exact title of page A (must exist) [required]
--b <str> Title to unlink from A; need not still exist as a page [required]
--dry-run Preview changes instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool xref remove --a "Gitea Actions" --b "act_runner"
tools/wikitool xref remove --a "Gitea Actions" --b "Act Runner" --dry-run
```
**Behaviour**
- Clears the reference in **both** directions: the cleanup command for a deleted or hand-renamed page rather than the strict inverse of a one-directional `xref add`.
- Clears `<B>` from every page-ref frontmatter field `<A>`'s type declares (`related:`, `sources:`, `entities:`, `concepts:`) plus the matching bullets.
- Also sweeps a page-ref field the type does *not* declare but some other type does, and drops that key once empty.
- `--b` need not still exist as a page, so this clears a reference left by a hand-deleted or hand-renamed page without hand-editing frontmatter.
- Idempotent: removing an absent link is a no-op. `--dry-run` reports without writing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - writes A then B, both idempotent
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Page A not found (B is allowed not to exist)
**See also**
- `wikitool xref add` - declares an edge
- `wikitool rm` - deletes a page and de-links it
## `xref link-source`
Batch-link a source page to every entity/concept it mentions.
**Synopsis**
- `wikitool xref link-source --source "Source - X" --entities A,B,C [--dry-run]`
**Options**
```text
--source <str> Exact source page title, e.g. 'Source - Docker Cheatsheet'
[required]
--entities <str> Comma-separated entity/concept titles the source mentions
[required]
--dry-run Preview which pages would be linked instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool xref link-source --source "Source - Docker Cheatsheet" --entities Docker,Podman --dry-run
tools/wikitool xref link-source --source "Source - Docker Cheatsheet" --entities Docker,Podman
```
**Behaviour**
- Each target gets the source page in its `sources:`, and the source page records each target in its own `entities:` or `concepts:`. No body bullet is written on either side - `sources:` *is* the record.
- Which field a target lands in follows its collection (`kb/entities/` -> `entities:`), so a new collection needs no code change.
- A target whose collection matches no reference field the source type declares is linked one way and named in the output.
- A target that does not exist is skipped and named; the others are still linked, and the run then exits 1.
- A target that is itself a source page is refused, named with the `cite add` command to use instead, and handled like a missing page: nothing is written for it, the others are still linked, and the run exits 1. `link-source` means "A mentions X" and writes `X.sources += A`, which for X = another source would claim the citation in the wrong direction; a citation between two sources has a direction only `cite add` records.
- Idempotent in both directions. `--dry-run` reports what would be linked.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - one write per entity plus one for the source page, idempotent per page
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Source page not found
- `1` - A page in `--entities` does not exist, or a target could not be written; the others were linked
- `1` - A page in `--entities` is itself a source page; the others were linked
- `1` - The source page itself could not be written after its targets were
**See also**
- `wikitool sources trace` - who a source is already linked to
- `wikitool xref add` - one labelled edge between two pages
- `wiki-ingest` skill - where a source is linked after it is written
## `links show`
Show the edges out of and into a page.
**Synopsis**
- `wikitool links show --page "<Title>" [--json]`
**Options**
```text
--page <str> Exact page title [required]
--json Print the edges as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool links show --page "Act Runner"
tools/wikitool links show --page "Act Runner" --json
```
**Behaviour**
- Shows the declared graph around one page in both directions: the edges it asserts (from its own `related:`, with labels) and the edges other pages assert about it.
- The inbound half is computed across the corpus on every call, never stored, so it is complete.
- `--json` prints both halves as JSON.
- Read-only; exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Page not found
**See also**
- `wikitool xref add` / `wikitool xref remove` - change the outbound edges
- `wikitool search` - finds the exact title
## `cite id`
Print the deterministic footnote id `cite add` would use for this (title, file) pair.
**Synopsis**
- `wikitool cite id --title "Source - X" [--file <qualifier>]`
**Options**
```text
--title <str> Source page title, e.g. 'Source - Docker Cheatsheet'
[required]
--file <str> Qualifier for a multi-file source, e.g. 'storage-model.md'
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool cite id --title "Source - Docker Cheatsheet"
```
**Behaviour**
- Prints the footnote id `cite add` would use for this (title, file) pair.
- A preview only: it does not check that the id is free on any given page.
- Never fails; safe to retry freely. Read-only and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool cite add` - writes the definition
## `cite add`
Upsert a `[^cite-id]: [[Source - X]]` definition in a page's footnotes region.
**Synopsis**
- `wikitool cite add --page "<Title>" --source "Source - X" [--file <qualifier>] [--dry-run]`
**Options**
```text
--page <str> Exact title of the page to add a citation on [required]
--source <str> Exact title of the source page being cited, e.g. 'Source - X'
[required]
--file <str> Qualifier for a multi-file source, e.g. 'storage-model.md'
--dry-run Preview instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool cite add --page "Docker" --source "Source - Docker Cheatsheet"
tools/wikitool cite add --page "Docker" --source "Source - Docker Cheatsheet" --file part-2
```
**Behaviour**
- Upserts a `[^cite-id]: [[Source - X]]` definition in the page's generated footnotes region, creating the region between `<!-- wikitool:footnotes -->` markers if absent.
- Reuses the id when the page already cites this exact source/file pair, so a repeat changes nothing.
- Adds `Source - X` to the page's frontmatter `sources:` - except on `Source - X` itself: a source page citing one of its own raw files (`--file`) gets the definition and leaves `sources:` untouched, since a page never lists its own title there.
- A source page may cite another source page this way - `cite add` is the command that records a citation between two sources, in the direction it was made.
- Prints the `[^cite-id]` marker; pasting it into the prose is a manual, editorial step.
- `--dry-run` reports without writing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - single file write
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - The page is not found
- `1` - The source page is not found - citing it would be a dangling reference
**See also**
- `wikitool cite sync` - prunes and re-orders the region
- `wikitool cite id` - previews an id
## `cite sync`
Reconcile each page's footnotes region against its actual `[^id]` references.
**Synopsis**
- `wikitool cite sync [--page "<Title>" | --all] [--dry-run]`
**Options**
```text
--page <str> Sync just this page
--all Sync every page under kb/
--dry-run Report what would change instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool cite sync --page "Docker"
tools/wikitool cite sync --all --dry-run
```
**Behaviour**
- Prunes definitions nothing references any more, re-renders the region in first-reference order, and reports every `[^id]` reference left with no definition.
- An undefined-reference report is not a failure: fix the reference, or run `cite add`, and re-run.
- A page still carrying the pre-4.0.0 undelimited footnote block is converted to a marked region in the same pass.
- Each page's re-render is idempotent; safe to retry freely. `--dry-run` reports without writing.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - one write per page, each idempotent
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Neither or both of `--page`/`--all` given, or the page is not found
- `1` - A page write failed partway
**See also**
- `wikitool cite add` - writes a definition
<!-- /wikitool:reference-commands -->
docs/reference/commands/pages.md
+532
View File
@@ -0,0 +1,532 @@
# Pages
Creating, updating, renaming, moving and removing wiki pages, and filing and closing items in the task tracker.
<!-- wikitool:toc -->
## Contents
- [`new`](#new)
- [`task new`](#task-new)
- [`task list`](#task-list)
- [`task close`](#task-close)
- [`touch`](#touch)
- [`rename`](#rename)
- [`rm`](#rm)
- [`move`](#move)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `new`
Scaffold a new wiki page of any type.
**Synopsis**
- `wikitool new <type-name> --name "<Name>" [--type <path>] [--set field=value ...]` - Any type; its type-spec decides fields, directory, title prefix and template.
- `wikitool new entity --name "<Name>" --set entity_type=<t> [--set tags=a,b] [--set related=X,Y] [--set sources="Source - Z"] [--set provenance=sourced|general|mixed]` - Writes `kb/entities/<subdir>/<Name>.md` - `<subdir>` from `entity_type` via the type-spec's `layout:`
- `wikitool new concept --name "<Name>" --set concept_type=<t> ...` - Writes `kb/concepts/<subdir>/<Name>.md` - `<subdir>` from `concept_type` via the type-spec's `layout:`
- `wikitool new source --name "<Name>" --set source_type=<t> --set raw_files=raw/notes/x.md,raw/notes/y.md --set fidelity=<f> --set authority=<a> [--set source_url=<URL>] [--set entities=A,B] [--set concepts=C,D]` - Writes `kb/sources/<subdir>/Source - <Name>.md` (prefix added automatically; `<subdir>` from `source_type` via the type-spec's `layout:`) with a `raw_files:` list; rejects paths that don't exist
- `wikitool new comparison --name "X vs Y" --set entities=X,Y` - Writes `kb/comparisons/X vs Y.md`
- `wikitool new project --name "<Name>" --set responsibility=<bereich> [--resume]` - Writes `kb/gtd/<bereich>/<Name>.md` and, with a task tracker configured, a same-named tracker project
**Options**
```text
Arguments:
type_name Type name, e.g. entity|concept|source|comparison (see `wikitool
types list`) [required]
Options:
--name <str> Page title (a type's title_prefix is added automatically)
[required]
--type <str> Override the type-spec path (defaults to the one named by
TYPE_NAME)
--set <str> Frontmatter field, repeatable: --set entity_type=tool --set
tags=a,b. Array values split on commas (escape a literal one as
\,); repeating --set for an array field appends instead of
replacing
--resume `project` only: confirm a human has completed the manual tracker
step an earlier HumanInterventionRequired refusal asked for, so
this run continues to page creation instead of refusing the now-
existing tracker project as a collision.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool new entity --name "Docker" --set entity_type=tool --set tags=containers
tools/wikitool new source --name "Docker Cheatsheet" --set raw_files=raw/2026/09/docker-cheatsheet.md --set fidelity=verbatim --set authority=reporting
tools/wikitool new project --name "Homelab migration" --set responsibility=infrastruktur --resume # re-run after exit 42, once the user created the tracker project
```
**Behaviour**
- A title becomes a file name, so it must be valid and unique on Windows and macOS as well as Linux, whichever platform runs the command and whichever root the type writes to. The rule is `kb/CONTRACT.md` § Titles are identifiers; it is checked on the full title, after `title_prefix`.
- The target's path below the instance root may be at most 160 characters, counted in UTF-16 code units the way Windows counts MAX_PATH, so a Windows checkout without long paths keeps working. A longer one is refused, for every root, naming the length and how much shorter it has to get.
- `new` never overwrites: a file already at the target - or one a case-insensitive file system would treat as the same file - is refused for every root, `instructions/` included.
- The type-spec drives everything: fields, directory (`base_dir`/`layout`), title prefix, and template. `types list`/`types describe` show what a type requires.
- The body skeleton is `types/<type>.<value>.md` when that file exists, `<value>` being the page's subtype field value (e.g. `types/entity.person.md` for `entity_type=person`) - it replaces the type-spec's `## Template` block whole. Without such a file, and for a type with no subtype field, the `## Template` block is used as before.
- A schema `default:` is materialized only for a field the schema also lists in `required:`.
- `--set` is repeatable, and comma-separated values fill array fields. An element that itself contains a comma is written `\,`, or passed as its own repeated `--set` for that field - repeating an array field appends.
- A capture field the type-spec requires (a source's `fidelity`/`authority`) must be passed with `--set`; `new` never guesses it and refuses `unknown` for it.
- Produces structurally correct frontmatter and a body skeleton only - the prose (Description, Summary, judgment calls about relationships) is written afterwards.
- `new project`: with `.wikitool-tasks.json` configuring a task tracker, also makes sure a same-named tracker project exists - one name, one identity. No tracker configured is a legitimate, explicitly announced state: page only.
- `new project`: tracker before page. The tracker side is settled first, so a failure past that point leaves a tracker project with no page - a state `review`'s check 3 reports - never a page with no tracker project.
- `new project`: a name already taken, case-insensitively, in `kb/` or the tracker is refused outright, naming where it was found, and creates nothing. For `caldav` the tracker check covers every list in the account, not only the ones counted as projects.
- `new project`: a provider whose configured access path has no write path (Super Productivity's `access: "snapshot"`) refuses entirely with exit 1, naming the `access: "api"` instance to use instead - neither the tracker project nor the page is created, and `--resume` behaves the same.
- `new project`: a provider that could write but has no project-creation call of its own (Super Productivity's `access: "api"` - `GET /projects` exists, `POST /projects` does not) prints instructions for a human and exits 42, creating nothing. That is not one of the named gates, but the same exit code and the same handling. `caldav` never does this: `MKCALENDAR` creates the list, so a valid, non-colliding name always creates it.
- `--resume` is how a later run tells the command a human has done what that message asked: it re-verifies through the tracker's read path before continuing to page creation, rather than trusting the claim, and exits 42 again if the tracker still does not have the project. `--resume` on any other type is refused.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: `new <type>`: Yes - single file write. `new project`: **No** for the tracker-configured case - a tracker-project write (or its human-clearance request) happens before the kb/ page write, so a failure between the two leaves a tracker project with no page (a state `review`'s check 3 already reports), never a page with no tracker project. Still a single file write when no tracker is configured
- Counts toward the session call budget: yes
- Can reach the network: no
- Gates: human-intervention-required (`new project` only) - see [exit codes and gates](../exit-codes.md)
**Exit codes**
- `0` - success
- `1` - A page with this title already exists, the type is unknown, or a `--set` value is invalid
- `1` - The title is not a valid file name (forbidden character, control character, reserved name such as `CON` or `Index`, trailing dot or space, empty), collides with another page by case or Unicode normalization, the target file already exists, or the target path is over the 160-character path budget
- `1` - A `raw_files` path does not exist
- `1` - A capture field the type-spec requires is missing, or set to `unknown`
- `1` - `--resume` with a type other than `project`
- `1` - new project: The name is already taken in the tracker (case-insensitively; for `caldav` against every list in the account)
- `1` - new project: The configured access path has no write path (Super Productivity's `access: "snapshot"`)
- `1` - new project: The page write failed after the tracker project was confirmed to exist
- `42` - new project: The provider cannot create the project itself (Super Productivity's `access: "api"`); the output says what a human has to create
**See also**
- `wikitool types describe <type>` - what a type requires and where it lands
- `wikitool touch` - changes a page's own frontmatter afterwards
- `wikitool task new` - a tracker item without a page
- `docs/explanation/knowledge-and-commitment.md` - why a project is a page and a tracker project
## `task new`
Create one open item in the configured task tracker - never a kb/ page.
**Synopsis**
- `wikitool task new --title "<Title>" (--project "<Name>" | --inbox) [--waiting [--follow-up-at YYYY-MM-DD]] [--notes "..."]`
**Options**
```text
--title <str> The item's title. Stored verbatim, never parsed.
[required]
--project <str> An existing tracker project's name (matched case-
insensitively). This command never searches or guesses
one and never creates one - use `wikitool new project`
first if it does not exist yet. Exactly one of
--project/--inbox is required.
--inbox File into the tracker's own inbox instead of a project -
the deliberately chosen exit when no project fits, never
a stand-in for an omitted --project. An item filed here
is invisible to `wikitool review`, since every check
there is reached through a project name and the inbox
has none.
--waiting Tag the item WAITING - the review's check 2 reads this.
--follow-up-at <str> YYYY-MM-DD. Only meaningful together with --waiting - it
is never a due date and is refused without --waiting.
--notes <str> A freetext backref, e.g. to the kb/ source page this
item came from. Stored verbatim, never parsed.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool task new --title "Renew the TLS certificate" --project "Homelab migration"
tools/wikitool task new --title "Quote from the electrician" --project "Homelab migration" --waiting --follow-up-at 2026-10-05
tools/wikitool task new --title "Read the qmd README" --inbox --notes "from [[Source - qmd - GitHub Repository]]"
```
**Behaviour**
- Creates one open item in the configured task tracker; never touches `kb/`.
- Exactly one of `--project` or `--inbox` is required; an omitted `--project` refuses rather than silently falling into the inbox.
- `--project` names an existing tracker project, matched case-insensitively - never created, and never searched or guessed.
- `--inbox` files into the tracker's own inbox. An item filed there never appears in `review`, since every one of its checks reaches items through a project name.
- `--waiting` sets the WAITING status `review`'s waiting-overdue check reads. `--follow-up-at` is refused without `--waiting` - it is never a due date on its own.
- `--notes` carries a freetext backref (e.g. to the `kb/` source page the item came from), stored verbatim, never parsed.
- The tracker configuration is read from `.wikitool-tasks.json`, or from the file `WIKITOOL_TASKS_CONFIG` names when that variable is set (so one checkout can be run against several trackers in turn). A set variable that names no file is an error, never "no tracker configured".
- No `.wikitool-tasks.json` fails immediately with a "no tracker configured" message.
- A provider whose configured access path has no write path (Super Productivity's `access: "snapshot"`) refuses entirely with exit 1, naming the `access: "api"` instance to use instead.
- **Never exits 42.** A `--project` matching no tracker project, or `--waiting` against a provider that cannot represent it right now (Super Productivity: the `waiting` tag does not exist yet, and its API cannot create tags), are ordinary exit-1 refusals that create nothing.
- Not idempotent: every successful run creates another item.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes - a single API call, made only once every precondition (the project's own id, the WAITING tag's own id) is confirmed to exist, so a missing one never leaves a half-written item behind
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - No `.wikitool-tasks.json` - no tracker configured
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
- `1` - Neither or both of `--project`/`--inbox`, or a `--follow-up-at` without `--waiting` or not `YYYY-MM-DD`
- `1` - A `--project` name matching no tracker project
- `1` - `--waiting` against a provider with no way to represent it right now (Super Productivity: the `waiting` tag does not exist)
- `1` - A read-only access path (Super Productivity's `access: "snapshot"`)
**See also**
- `wikitool task list` - a project's open items and their ids
- `wikitool task close` - marks an item done
- `wikitool new project` - a project page and its tracker project
- `docs/explanation/knowledge-and-commitment.md` - why commitments live in the tracker, not in `kb/`
## `task list`
List a project's open items - id, title, and whether each carries the WAITING status.
**Synopsis**
- `wikitool task list --project "<Name>"`
**Options**
```text
--project <str> An existing tracker project's name (matched case-
insensitively). [required]
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool task list --project "Homelab migration"
```
**Behaviour**
- Lists one tracker project's open items: id, title, and whether each carries the WAITING status.
- The id source `task close` and `review`'s `waiting_overdue`/`someday_stale` findings need, without running `review` first.
- Works on every access path a provider offers, read-only ones included.
- A `--project` matching no tracker project prints "No open items": the tracker read does not distinguish an empty project from an unknown one.
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Yes - read-only, nothing to leave half-written
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `0` - A `--project` matching no tracker project - prints "No open items", not an error
- `1` - No `.wikitool-tasks.json` - no tracker configured
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
**See also**
- `wikitool task close` - closes an item by the id listed here
- `wikitool review` - the findings that name items
## `task close`
Mark one tracker item done - never delete it.
**Synopsis**
- `wikitool task close --id <item-id>`
**Options**
```text
--id <str> The tracker's own item id, e.g. from `task list` or `wikitool
review`'s waiting_overdue/someday_stale findings - never a title.
[required]
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool task close --id <item-id>
```
**Behaviour**
- Marks one tracker item done. It never deletes or moves an item - the only closing write this stack makes.
- `--id` is the provider's own item id, from `task list` or a `review` finding - never a title.
- The tracker configuration is read from `.wikitool-tasks.json`, or from the file `WIKITOOL_TASKS_CONFIG` names when that variable is set (so one checkout can be run against several trackers in turn). A set variable that names no file is an error, never "no tracker configured".
- No `.wikitool-tasks.json` fails with a "no tracker configured" message.
- A provider whose configured access path has no write path (Super Productivity's `access: "snapshot"`) refuses entirely with exit 1, naming the `access: "api"` instance to use instead.
- **Never exits 42.**
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - a single API call; an unknown id is rejected by the provider itself (Super Productivity: `404 TASK_NOT_FOUND`) before anything is written
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - No `.wikitool-tasks.json` - no tracker configured
- `1` - `WIKITOOL_TASKS_CONFIG` names a file that does not exist or is broken
- `1` - An `--id` matching no tracker item right now
- `1` - A read-only access path (Super Productivity's `access: "snapshot"`)
**See also**
- `wikitool task list` - where the id comes from
- `wikitool review` - findings that carry item ids
## `touch`
Bump a page's `modified:` date and optionally rewrite its other frontmatter fields.
**Synopsis**
- `wikitool touch --page "<Title>" [--summary "..."] [--provenance <v>] [--date YYYY-MM-DD] [--set field=value ...] [--add field=value ...] [--remove field=value ...] [--no-date] [--dry-run]`
**Options**
```text
--page <str> Exact page title, e.g. 'Docker Cheatsheet' [required]
--summary <str> Replace the page's 1-line summary
--provenance <str> Replace the page's provenance marker
(sourced|general|mixed)
--date <str> Date to record (YYYY-MM-DD). `modified:` defaults to
today; a source's `date:` is its publication date and
changes only when given here.
--set <str> Replace a frontmatter field, repeatable: --set tags=a,b.
Array values split on commas (escape a literal one as \,);
repeating --set for one array field appends within this
call. Page-reference fields belong to `xref`, not here
--add <str> Append elements to an array field without naming the whole
list: --add tags=x. Elements already present are left
alone
--remove <str> Drop elements from an array field: --remove tags=x.
Removing an absent element succeeds and says so
--no-date Only change the given fields; leave the modified/date
field alone
--dry-run Preview the new frontmatter instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool touch --page "Docker" --summary "Container runtime and image format"
tools/wikitool touch --page "Docker" --add tags=containers --no-date
tools/wikitool touch --page "Source - Docker Cheatsheet" --date 2026-09-01 --dry-run
```
**Behaviour**
- Bumps `modified:` to today and optionally rewrites any other field the page's type declares.
- `--summary`/`--provenance` are shorthands; `--set` reaches every other field and **replaces** its value, while `--add`/`--remove` change single elements of an array field (removing an absent element succeeds and says so).
- Repeating `--set` for one array field appends *within the call*, and `\,` is a literal comma.
- Refused, naming the command that owns them instead: `type:` (the page-lifecycle procedure) and the page-ref arrays `related:`/`sources:`/`entities:`/`concepts:` (`xref`). Everything else the schema declares is settable, and an unknown field is refused with the list of fields the page actually has.
- Schema-validates the fields it writes; `raw_files:` entries must exist on disk.
- A source page declares `date:` instead of `modified:`: the *publication* date of the raw material. It is never bumped to today and changes only when `--date` names a value explicitly.
- `--no-date` changes only the given fields and leaves the date alone; `--dry-run` previews the new frontmatter without writing.
- Safe to re-run as-is: `--set` and `--add` are idempotent, and `--remove` of an already-absent element succeeds while reporting it.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - single file write, and every refusal happens before it
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Page not found
- `1` - An invalid value for a field it writes, or a `raw_files:` path that doesn't exist
- `1` - A field owned by another command (`type:`, a page-ref array) or absent from the type's schema
- `1` - `--add`/`--remove` on a non-array field
**See also**
- `wikitool xref add` / `wikitool xref remove` - the page-ref arrays
- `instructions/page-lifecycle.md` - changing a page's `type:`
## `rename`
Rename a page, or repoint references that name a page that never existed.
**Synopsis**
- `wikitool rename --from "<Old>" --to "<New>" [--dry-run]`
**Options**
```text
--from <str> Current page title, exactly as it appears [required]
--to <str> New page title [required]
--dry-run List what would change instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool rename --from "act_runner" --to "Act Runner" --dry-run
tools/wikitool rename --from "Docker Engine" --to "Docker"
```
**Behaviour**
- Renames a page and repoints every reference to it: body `[[wikilinks]]` (aliases and anchors preserved), a `[^cite-id]` whose id was derived from the old title (refreshed to match the new one, both in its Footnotes definition and every reference to it), the page's own H1, and every page-ref frontmatter array declared by the type's `page_ref_fields:`.
- If `--from` is *not* a page but is referenced, it instead repoints those references onto the existing `--to` page and moves nothing - the fix for a reference spelled `act_runner` when the page is `Act Runner`.
- Each page's rewrite is idempotent, so a re-run as-is is safe. If a write fails midway, nothing is renamed on disk and the error lists what was updated.
- `--dry-run` lists every page it would change; run it first to see the blast radius.
- Only `--to` is checked against the title rule and the path budget (160 UTF-16 code units for the whole path below the instance root). A page whose current title breaks either (`lint`'s Unportable Titles and Long Paths) can always be renamed away from it, and a title that differs from the page's own only by case (`Foo` to `FOO`) is allowed.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - one write per referencing page, then the file move
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `--from` equals `--to`
- `1` - Neither `--from` nor `--to` is a page
- `1` - The `--to` title is already taken - also by a page that differs only in case or Unicode normalization, or by a file in the page's directory - or is not a valid file name, or would put the page's path over the path budget (see `kb/CONTRACT.md` § Titles are identifiers)
- `1` - A page write failed partway; nothing was renamed on disk
**See also**
- `instructions/page-lifecycle.md` - renaming, moving and deleting a page
- `wikitool move` - changes a page's directory, not its title
- `wikitool rm` - deletes a page
## `rm`
Delete a page and mechanically de-link it from the rest of the wiki.
**Synopsis**
- `wikitool rm --page "<Title>" [--yes] [--dry-run]`
**Options**
```text
--page <str> Exact title of the page to delete [required]
-y, --yes Confirm deletion of a page that other pages still reference.
Only pass this after a human has reviewed the inbound list -
never set it automatically.
--dry-run List what would change instead of writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool rm --page "Old Draft" --dry-run
tools/wikitool rm --page "Old Draft" --yes # after the user approved the inbound list
```
**Behaviour**
- Deletes a page and de-links it from the rest of the wiki: strips page-ref array entries and bare `- [[Title]]` / `- **label:** [[Title]]` bullets pointing at it.
- Leaves prose references and inline citations in place and reports them afterwards; those are an editorial fix, not a reason to retry.
- Refuses without `--yes` while other pages still reference it, listing them.
- If a de-link write fails midway, the page is not deleted, so nothing is orphaned and a re-run is safe.
- `--dry-run` lists what would change, without writing.
- Not idempotent: once the page is gone, a second run finds nothing to delete.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - one write per referencing page, then the delete
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Page not found
- `1` - Other pages still reference it and `--yes` was not passed
- `1` - A de-link write failed partway; the page was not deleted
**See also**
- `instructions/page-lifecycle.md` - when a page is deleted rather than renamed
- `wikitool rename` - repoints references instead of removing them
## `move`
Move a page (or every misplaced page) to the directory its type-spec computes.
**Synopsis**
- `wikitool move --page "<Title>"`
- `wikitool move --reconcile [--dry-run]`
**Options**
```text
--page <str> Exact title of the page to move to its computed location
--reconcile Move every page under kb/ that is not at its computed location
--dry-run List what would move without writing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool move --page "Docker"
tools/wikitool move --reconcile --dry-run
tools/wikitool move --reconcile
```
**Behaviour**
- Moves a page to the directory its type-spec computes for its current frontmatter (`base_dir` + `layout`, the same rule `new` places a page by) - never to a hand-chosen destination; there is no `--to <dir>`.
- A destination whose path would be over the path budget (160 UTF-16 code units below the instance root) is refused, and `--reconcile` skips such a page and names it, as it does for an occupied destination - `wikitool rename` the page to a shorter title.
- `--reconcile` applies it corpus-wide: every misplaced page moves in one call, and a second run reports nothing left to do. It fixes `lint`'s `Misplaced Pages` (advisory) and `Nested Pages` (hard) findings.
- Neither mode touches a body or a frontmatter field, and the page's title - its only identity in the wiki - never changes; only the file moves.
- A directory a move empties is removed with it, so a page nested below its area leaves no leftover directory behind.
- A page already at its computed location is reported and left alone, and `--reconcile` only re-moves what is still misplaced, so a re-run is safe.
- `--dry-run` (with `--reconcile`) lists the moves; run it first to see the blast radius.
- Run `wikitool index rebuild` afterwards.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: `--page`: yes, a single file move. `--reconcile`: no - one file move per page, each idempotent
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Neither or both of `--page`/`--reconcile` given
- `1` - The named page is not found, or has no `type:` to compute a placement from
- `1` - The destination already holds an entry with the same name, or one that differs only in case or Unicode normalization (a pre-existing duplicate-stem collision), or its path would be over the path budget - refused rather than silently skipped
- `1` - `--reconcile` failed partway
**See also**
- `wikitool lint` - reports Misplaced and Nested Pages
- `wikitool index rebuild` - run after moving
- `instructions/page-lifecycle.md` - moving, renaming and deleting a page
<!-- /wikitool:reference-commands -->
docs/reference/commands/provenance.md
+151
View File
@@ -0,0 +1,151 @@
# Provenance
Which raw source backs which page, and the generated provenance index that records it.
<!-- wikitool:toc -->
## Contents
- [`sources coverage`](#sources-coverage)
- [`sources trace`](#sources-trace)
- [`sources rebuild-index`](#sources-rebuild-index)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `sources coverage`
List raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages.
**Synopsis**
- `wikitool sources coverage [--json]`
**Options**
```text
--json Print raw findings as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool sources coverage
tools/wikitool sources coverage --json
```
**Behaviour**
- Lists raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages.
- `--json` prints the same lists as JSON.
- Never fails; read-only and safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool sources trace` - follows one file or page
- `wiki-ingest` skill - turns an uncovered raw file into a source page
## `sources trace`
Trace provenance in either direction: raw file, or page.
**Synopsis**
- `wikitool sources trace --raw <path> | --page "<Title>"`
**Options**
```text
--raw <str> Raw file path to trace forward from
--page <str> Wiki page title to trace backward from
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool sources trace --page "Docker"
tools/wikitool sources trace --raw "raw/articles/llm-wiki.md"
```
**Behaviour**
- `--raw <path>`: raw file -> the source page(s) covering it -> the pages citing those.
- `--page "<Title>"`: page -> its sources -> their raw files.
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Neither or both of `--raw`/`--page` given, or `--page` names an unknown page
- `1` - `--raw` names a file no source page covers - reported as a plain finding plus exit 1, not the usual `ERROR`-prefixed rejection
**See also**
- `wikitool sources coverage` - every uncovered raw file at once
- `wikitool xref link-source` - links a source page to what it mentions
## `sources rebuild-index`
Regenerate the `kb/provenance.md` reverse index.
**Synopsis**
- `wikitool sources rebuild-index [--dry-run]`
**Options**
```text
--dry-run Print the result instead of writing kb/provenance.md
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool sources rebuild-index
tools/wikitool sources rebuild-index --dry-run
```
**Behaviour**
- Regenerates the `kb/provenance.md` reverse index (raw file -> source page -> citing pages) from scratch.
- `--dry-run` prints the result instead of writing `kb/provenance.md`.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - the single provenance file is regenerated from scratch
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - An I/O error while writing `kb/provenance.md` (rare)
**See also**
- `wikitool index rebuild` - the page catalog, rebuilt alongside
- `instructions/publish-cycle.md` - where a write session runs this
<!-- /wikitool:reference-commands -->
docs/reference/commands/raw-material-and-uploads.md
+558
View File
@@ -0,0 +1,558 @@
# Raw material and uploads
Bringing new material into `raw/`: fetching and capturing it, seeing what is waiting, accepting it, and reviewing what an MCP client uploaded.
<!-- wikitool:toc -->
## Contents
- [`raw fetch`](#raw-fetch)
- [`raw capture`](#raw-capture)
- [`raw status`](#raw-status)
- [`raw pending`](#raw-pending)
- [`raw accept`](#raw-accept)
- [`upload list`](#upload-list)
- [`upload show`](#upload-show)
- [`upload accept`](#upload-accept)
- [`upload reject`](#upload-reject)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `raw fetch`
Capture a web page the user names into `incoming/`: the HTML as received plus a derived text, for `raw accept` to promote.
**Synopsis**
- `wikitool raw fetch <url> [--name <stem>]` - Fetch the page and write `incoming/<stem>.html` and `incoming/<stem>.md`
- `wikitool raw fetch --html incoming/<file>.html --url <url>` - Derive the `.md` from a page a human saved from their browser - no network access
**Options**
```text
Arguments:
url The http(s) URL to fetch
Options:
--html <path> Derive the text from this HTML file under incoming/ (saved from
a browser) instead of fetching - no network access
--url <str> With --html: the page's URL, for the header and relative links
--name <str> File stem in incoming/ instead of the one computed from the URL
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw fetch https://example.org/blog/post
tools/wikitool raw fetch https://example.org/ --name example-start
tools/wikitool raw fetch --html incoming/post.html --url https://example.org/blog/post
```
**Behaviour**
- Writes into `incoming/` only, never into `raw/`: `raw accept` promotes both files afterwards, in one call, as one bundle under `raw/<YYYY>/<MM>/<stem>/`.
- The `.html` holds the response body byte for byte. The `.md` starts with a fixed header block (`fetched_by`, `url`, `final_url`, `retrieved`, `http_status`, `content_type`, `charset`, `title`, `derived_from`) followed by the derived text.
- Character set, in this order: byte-order mark, the HTTP `Content-Type` charset, a `<meta>` declaration in the first 4 KiB, UTF-8. Undecodable bytes are replaced with U+FFFD and reported; the header names the charset and where it came from.
- Derived text: the content root is `<main>`, else the single `<article>`, else `<body>`; `script`, `style`, `noscript`, `nav`, `header`, `footer`, `aside`, `form`, `template` and `svg` are dropped below it. Headings, lists, code, tables and links (made absolute) become Markdown. The same bytes always give the same text.
- The stem is the URL's last path segment without its extension, else its host name - ASCII, lowercase, `-`-separated, at most 60 characters; `--name` overrides it.
- A `text/plain` or `text/markdown` response is stored as received as `.txt`/`.md`; any other non-HTML response (PDF, image, ...) as received under the extension of its content type. Neither gets a derived file or a header.
- Only `http`/`https`, on every redirect hop too. 30 s for the whole transfer, 25 MiB at most. No cookies, no JavaScript: a derived text under 200 characters is written, with a warning to read it before accepting.
- `--html` derives the `.md` beside a page saved to `incoming/` from a logged-in browser - the way past a paywall or a script-rendered page. Its header carries `derived` (when the text was derived) instead of `retrieved`, `final_url`, `http_status` and `content_type`, and the `.html` is left untouched.
- Success prints the `raw accept` line for the written files and the `source_url` for the source page.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes for what it leaves behind - one or two new files in `incoming/`, created exclusively; a failure on the second removes the first
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - The URL is not `http`/`https` (also after a redirect), or neither or both of `<url>` and `--html` were given
- `1` - A target file already exists in `incoming/`
- `1` - The server answered with an HTTP error, could not be reached, took longer than 30 s, or sent more than 25 MiB
- `1` - raw fetch --html: `--url` is missing, the file is not directly in `incoming/` or does not exist, or `incoming/<stem>.md` already exists
**See also**
- `raw/CONTRACT.md` "Getting a URL in: `raw fetch`" - the rules and why
- `wikitool raw accept` - promotes the written files into `raw/`
- `instructions/wiki-ingest/SKILL.md` - where a URL to ingest starts
## `raw capture`
Capture documentation from a git repository into `incoming/<bundle>/`, with a manifest naming the repository, ref rule and commit, for `raw accept` to promote.
**Synopsis**
- `wikitool raw capture <repo-url> --ref <branch|tag-pattern> --path <glob> [--path <glob> ...] --name <bundle> --fidelity <v> --authority <v>` - First capture of a repository
- `wikitool raw capture --update <raw-bundle> [--fidelity <v>] [--authority <v>]` - Capture the current state of an already-accepted bundle, for `raw accept --replaces-bundle`
**Options**
```text
Arguments:
repo_url The repository URL (ssh://, https:// or user@host:path)
Options:
--ref <str> A branch name (main) or a tag pattern (v*)
--path <str> A glob over the repository's paths; repeatable
--name <str> The bundle name - the folder in incoming/ and under raw/
--fidelity <str> How faithful the capture is (see `wikitool types describe
source`)
--authority <str> What the material may claim about its subject (see
`wikitool types describe source`)
--update <path> Capture the current state of this captured bundle under
raw/, from its own manifest
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw capture ssh://git@example.org/team/service.git --ref main --path 'docs/**/*.md' --path README.md --name service-docs --fidelity verbatim --authority normative
tools/wikitool raw capture https://example.org/team/lib.git --ref 'v*' --path docs --name lib-docs --fidelity verbatim --authority reporting
tools/wikitool raw capture --update raw/2026/10/service-docs
```
**Behaviour**
- Writes into `incoming/<bundle>/` only, never into `raw/`: `raw accept incoming/<bundle>` promotes it, as one folder, to `raw/<YYYY>/<MM>/<bundle>/`.
- `--ref` is a branch name (`main`) or a tag pattern (`v*`, any of `*?[`), which names the newest matching tag by version order. It resolves to one commit - for an annotated tag, the commit it points at.
- The commit is fetched by ref name, shallowly, into a bare cache repository per URL under `tools/.wikitool_capture/` (gitignored). Files are read from it as blobs, never through a checkout, so they are byte-identical to the repository's - no line-ending conversion, no filter.
- `--path` globs (repeatable) select files by their path in the repository, as git's `:(glob)` pathspec does: `*`, `?` and `[...]` do not cross `/`; `**` as a whole segment spans any number of directories; a glob without wildcards also takes everything below it as a directory. Files keep their repository paths inside the bundle.
- Never captured, each named in the output with its reason: a file whose first line starts with `<!-- wikitool:export`, a symlink, a submodule, a file over 25 MiB, a path with a segment starting with `.`, a file named `_capture.json`, and a Git LFS pointer.
- `incoming/<bundle>/_capture.json` records `schema`, `repo`, `ref`, `commit`, `paths`, `captured` (UTC), `fidelity`, `authority` and `files`. It is the only declaration that and how this instance follows the repository.
- Only `ssh://`, `https://` and the scp form `user@host:path`; a URL with a password or token in it is refused, since the manifest is committed. Git runs with the host's own credentials and configuration, restricted to those protocols, and never prompts: a repository that asks for credentials is reported as unreachable. Each git call times out after 120 s.
- `--update <raw-bundle>` takes URL, ref rule and globs from that bundle's manifest and writes the current state to `incoming/<same bundle name>/`; `--fidelity`/`--authority` default to the manifest's values, and passing one is the way to correct it.
- Refused before anything is written: `incoming/<bundle>` already exists, the name is already taken under `raw/` (a first capture), nothing matches the globs, or a target path - measured at its later place under `raw/` - is over the path budget.
- Success prints the commit and the `raw accept` line that comes next.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes for what it leaves in `incoming/` - the bundle is written into a hidden staging directory and renamed into place in one step; a failure removes the staging directory. The git cache under `tools/.wikitool_capture/` is updated either way
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `1` - The URL is not `ssh://`, `https://` or `user@host:path`, or carries a password; the `--ref` rule, a `--path` glob or `--name` is unusable; or an argument of the other variant was mixed in
- `1` - `--fidelity`/`--authority` is missing on a first capture, or names `unknown` or a value outside the schema's enum
- `1` - The repository could not be reached, asked for credentials, or timed out; or no branch or tag matches `--ref`
- `1` - `incoming/<bundle>` already exists, the name is already taken under `raw/`, nothing matches the globs, or a path would be over the budget
- `1` - raw capture --update: The path is not a bundle under `raw/` with a readable `_capture.json`, or its manifest names a URL that is refused
**See also**
- `raw/CONTRACT.md` "Getting a repository in: `raw capture`" - the rules and why
- `wikitool raw accept` - promotes the bundle; `--replaces-bundle` for a new edition
- `wikitool raw status` - which captured bundles have fallen behind their repository
## `raw status`
Report which captured bundles under `raw/` have fallen behind their repository, with each changed file as `A`/`M`/`D`.
**Synopsis**
- `wikitool raw status [--json]`
**Options**
```text
--json Print one JSON object per captured bundle
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw status
tools/wikitool raw status --json
```
**Behaviour**
- Reads every `_capture.json` under `raw/` and resolves its ref rule with `git ls-remote`. Only where the commit moved is the new commit fetched, and compared - with the same globs and exclusions `raw capture` applies - against the bundle's files under `raw/`.
- A bundle whose commit moved without a change inside its globs is reported as unchanged, not listed. A tag pattern follows new matching tags only, never new commits on a branch.
- Each changed bundle prints `old -> new` and its files as `A`/`M`/`D`, grouped by owning source page with its citing pages, followed by the `raw capture --update` and `raw accept --replaces-bundle` lines that take the new edition in.
- An unreachable repository, a refused URL or an unreadable manifest is one line beside the others, never an abort; the exit status is 0 as long as the command itself ran.
- `--json` prints one object per bundle: `bundle`, `repo`, `ref`, `old`, `new`, `changed`, `files` (`[{path, status}]`), `error` (or null).
- Git runs exactly as for `raw capture`: the host's credentials, only `ssh`/`https`, never a prompt, 120 s per call.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only for the tree - writes nothing but the git cache under `tools/.wikitool_capture/`
- Counts toward the session call budget: yes
- Can reach the network: yes
**Exit codes**
- `0` - success
- `0` - A repository could not be reached, its manifest or URL was refused, or no ref matches its rule
**See also**
- `wikitool raw capture` - `--update` captures the new edition
- `wikitool raw accept` - `--replaces-bundle` takes it in
- `raw/CONTRACT.md` "Getting a repository in: `raw capture`"
## `raw pending`
List what waits in `incoming/`, oldest first, and name the entry an ingest without an argument takes next.
**Synopsis**
- `wikitool raw pending [--json]`
**Options**
```text
--json Print the candidates as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw pending
tools/wikitool raw pending --json
```
**Behaviour**
- A candidate is a top-level entry of `incoming/`: a single `file`, a `bundle` of top-level files sharing a stem (a `raw fetch` pair, a PDF and its converted text), or a `folder` with every file below it. Dotfiles, empty directories and their contents are none; `mcp-upload/` is outside `incoming/` and never listed.
- Order: oldest first by modification time. A bundle or folder counts as new as its newest file; a tie goes by name. The mtime is when a document last changed only if it was copied with its timestamps kept (`cp -p`, `rsync -a`, an unpacked archive) - for a download or a `raw fetch` it is merely when it was dropped.
- Each candidate shows its path(s), kind, file count and mtime, and whether `raw accept` would take it as it stands - the same checks, minus `--fidelity`/`--authority`. One it would refuse is listed with the reason and skipped: it needs a human.
- The default is the first candidate `raw accept` would take, and the output names it.
- `--json` prints the same candidates in the same order: `kind`, `paths`, `files`, `mtime`, `acceptable`, `reason`, `default`.
- Reads directory listings and `lstat` only, never a file's content; an empty `incoming/` is exit 0 with nothing to do.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool raw accept` - promotes the chosen candidate
- `instructions/wiki-ingest/SKILL.md` - ingest without an argument starts here
- `raw/CONTRACT.md` "Getting a file in: incoming/" - candidates and order, and why
## `raw accept`
Promote one or more files, or one folder, from `incoming/` into `raw/`.
**Synopsis**
- `wikitool raw accept <file> [<file> ...] --fidelity <v> --authority <v> [--page "<Title>"] [--dry-run]` - Promote one or more files from `incoming/` into today's `raw/<YYYY>/<MM>/` shard
- `wikitool raw accept incoming/<folder> --fidelity <v> --authority <v> [--dry-run]` - Promote a whole folder as one source, its structure kept, into `raw/<YYYY>/<MM>/<folder>/`
- `wikitool raw accept <file> --replaces <raw-path> [--fidelity <v>] [--authority <v>] [--dry-run]` - Overwrite one existing raw file in place with a new edition
- `wikitool raw accept incoming/<bundle> --replaces-bundle <raw-bundle> [--dry-run]` - Replace a captured bundle as a whole with the new edition `raw capture --update` wrote
**Options**
```text
Arguments:
files... One or more files directly in incoming/, all belonging to the same
source - or one folder in incoming/, accepted whole as one source
[required]
Options:
--fidelity <str> How faithful the capture is (see `wikitool types
describe source`). Required unless --replaces is
given, where it is the optional, sanctioned way to
correct an already-set value
--authority <str> What the material may claim about its subject (see
`wikitool types describe source`). Required unless
--replaces is given, where it is the optional,
sanctioned way to correct an already-set value
--page <str> Extend this existing source page's raw_files: with
the promoted file(s), folding in its already-
promoted file if this raises it past one
--replaces <path> Overwrite this existing raw/ file in place with the
single incoming/ file passed alongside it - the only
sanctioned way past the stem-uniqueness rule below
--dry-run List what would move without writing
--replaces-bundle <path> Replace this captured bundle under raw/ as a whole
with the new edition in the incoming/ folder passed
alongside it (written by raw capture --update)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool raw accept incoming/docker-cheatsheet.md --fidelity verbatim --authority reporting
tools/wikitool raw accept incoming/part-2.md --fidelity verbatim --authority reporting --page "Source - Docker Cheatsheet"
tools/wikitool raw accept incoming/projekt-x --fidelity verbatim --authority reporting
tools/wikitool raw accept incoming/cluster.md --replaces raw/documents/cluster.md
tools/wikitool raw accept incoming/chemenu --replaces-bundle raw/2026/10/chemenu
```
**Behaviour**
- Promotes files from `incoming/` into `raw/<YYYY>/<MM>/`, computed from the accept date rather than chosen by hand. A file argument must sit directly in `incoming/`; a file inside a subdirectory is refused, since a subdirectory is a source of its own.
- A folder argument (`incoming/<folder>`) is one source: every file below it moves to `raw/<YYYY>/<MM>/<folder>/` at the same relative path, and the directories left empty are removed - `incoming/<folder>` no longer exists afterwards. The folder name is the bundle name; two `README.md` in different subfolders are no conflict.
- A folder is accepted alone - no other argument, no `--page`, no `--replaces` - with one `--fidelity`/`--authority` pair for all of it. It is refused, before anything moves, if it is empty, or if a hidden entry (name starting with `.`), a symlink or a special file sits anywhere below it; the refusal names each one.
- One file promoted alone lands with no directory of its own; several files in one call nest under `raw/<YYYY>/<MM>/<stem>/`, named after the first file's stem.
- `--fidelity`/`--authority` are required on a plain accept (`types describe source` lists the values); `unknown` is refused - it is backfill-only.
- `--page "<Title>"` additionally extends that existing source page's `raw_files:` in the same call and writes both capture fields onto it - refused if it already carries a different value, since a capture field is fixed once.
- If `--page` raises the page past one file, its already-promoted file is folded into a bundle at *its own* parent directory, not today's shard, so a bundle never mixes an old and a new capture date - after checking that file has no other owner.
- Every name occupied anywhere under `raw/` - file stems and bundle directory names alike, old type directories and date shards together - stays unique: a promote whose target name already belongs to something this call does not itself own is refused, naming `--replaces` and renaming in `incoming/` as the two routes, without recommending either.
- `--replaces <raw-path>` overwrites that file in place with the single incoming file (same filename required), leaves every page's `raw_files:` untouched and writes no `kb/` page; the previous edition survives only in `git log --follow <raw-path>`.
- With `--replaces`, `--fidelity`/`--authority` are optional, and passing one overwrites the owning page's already-set value - the one path the fixed-once rule does not block.
- `--replaces` refuses a target with more than one owning source page; with none, it replaces anyway and says so. It cannot be combined with `--page` or with more than one incoming file.
- `--replaces` prints the source page (if any) and its citing pages, so their update lands in the same commit as the replacement.
- A file already at its computed destination is what "already exists" reports, not a partial prior run to resume - safe to retry as-is once a cause is fixed.
- A folder carrying `_capture.json` at its top is a captured bundle (`raw capture`): `--fidelity`/`--authority` come from that manifest only and are refused on the command line; the folder must hold exactly the files the manifest lists. `_capture.json` goes in no `raw_files:`.
- `--replaces-bundle <raw-bundle>` replaces a captured bundle under `raw/` with the new edition in `incoming/<bundle>` at the same address: files gone from the new edition are removed, emptied directories `rmdir`ed, and afterwards the bundle holds exactly the new manifest's files plus `_capture.json`. `raw_files:` is left untouched; an owning source page's `fidelity`/`authority` is overwritten where the new manifest differs.
- `--replaces-bundle` prints each changed file as `A`/`M`/`D`, grouped by owning source page with its citing pages, and the `touch --page ... --add/--remove raw_files=` lines that follow; `git diff` on the bundle shows the `M` and `D` files, the `A` files appear only in that list because they are still untracked.
- `--replaces` and `--page` refuse a target inside a captured bundle; the refusal names `raw capture --update` and `--replaces-bundle`. A file named `_capture.json` anywhere but at a captured folder's top is refused - the name is reserved.
- `--dry-run` reports the moves without making them.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: `raw accept`: No - one filesystem move per file, then (with `--page`) one page write. With a folder: No - one move per file, then one `rmdir` per emptied directory; a half-accepted folder is not resumed. `raw accept --replaces`: No - one `unlink()` + one `rename()`, plus (if `--fidelity`/`--authority` was given) one page write. `raw accept --replaces-bundle`: No - one `unlink()` per removed file, one `rename()` per added or modified file, then the manifest, then one `rmdir` per emptied directory, plus one page write per owning page whose capture fields change; a half-replaced bundle is not resumed
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - raw accept: A file does not exist or is not directly in `incoming/`; two files in one call share a filename; a target path already exists; or a target path would be over the path budget (160 UTF-16 code units below the instance root)
- `1` - raw accept incoming/<folder>: The folder is not directly in `incoming/`, is combined with another argument, `--page` or `--replaces`, holds no file, or holds a hidden entry, a symlink or a special file; a target path is over the budget; or the folder name is already occupied under `raw/`
- `1` - A file - given directly, inside a folder, or as `--replaces`/`--replaces-bundle` material - is a guideline export: its first line, after an optional BOM, starts with `<!-- wikitool:export`
- `1` - raw accept: `--fidelity`/`--authority` is missing, or names `unknown` or a value outside the schema's enum
- `1` - raw accept: The target name is already occupied anywhere under `raw/` by something the call does not own
- `1` - raw accept: `--page` names an unknown page or one with no `raw_files:` yet, an existing `raw_files:` entry is missing on disk, a file to be moved has more than one owning page, or `--page` would overwrite an already-set `fidelity`/`authority` with a different value
- `1` - raw accept --replaces: More than one incoming file, or `--page` also given
- `1` - raw accept --replaces: The incoming file does not exist or is not directly in `incoming/`, its filename differs from the target's, or the target does not lie under `raw/` or does not exist
- `1` - raw accept --replaces: `--fidelity`/`--authority` names `unknown` or a value outside the schema's enum, or the target has more than one owning source page
- `1` - raw accept --replaces / --page: The target, or a file the page already has, lies in a captured bundle
- `1` - raw accept incoming/<bundle>: A captured folder is given `--fidelity`/`--authority`, its `_capture.json` cannot be read or names an invalid value, or its files differ from the ones the manifest lists
- `1` - raw accept --replaces-bundle: Either side has no `_capture.json`, the two manifests name different repositories, the folder's name differs from the bundle's, `--page`, `--replaces`, `--fidelity` or `--authority` was also given, the folder does not match its manifest, a new path is over the budget, or a file of the bundle has more than one owning source page
**See also**
- `raw/CONTRACT.md` "Getting a file in: incoming/" - the rules and why
- `wikitool raw pending` - what is waiting in `incoming/`, and which entry is next
- `wikitool raw capture` - writes a captured bundle, or its new edition, into `incoming/`
- `wikitool types describe source` - the capture field values
- `wikitool new source` - the source page for a promoted file
## `upload list`
List every MCP submission currently waiting in the quarantine (`mcp-upload/`).
**Synopsis**
- `wikitool upload list [--json]`
**Options**
```text
--json Print every waiting submission as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool upload list
tools/wikitool upload list --json
```
**Behaviour**
- Lists every submission waiting in `mcp-upload/`, oldest id first: id, filename, size, submitter.
- Only ever non-empty when `.wikitool-upload.json` opts the checkout into the MCP server's `submit` tool.
- A submission directory with a corrupt manifest is silently skipped.
- Never fails; read-only and safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool upload show` - one submission's manifest
- `INSTALL-MCP.md` § "Schritt 7: Optional - den `submit`-Pfad freischalten"
## `upload show`
Print one submission's manifest in full.
**Synopsis**
- `wikitool upload show <id> [--json]`
**Options**
```text
Arguments:
submission_id A submission id from `upload list` [required]
Options:
--json
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool upload show <id>
```
**Behaviour**
- Prints one submission's manifest in full: filename, size, sha256, submitter, submitter source (the header name, not a claim the header was honest), submission time.
- What a reviewer reads before `upload accept`.
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unknown or malformed submission id
**See also**
- `wikitool upload accept` - promotes it after review
- `wikitool upload reject` - declines it
## `upload accept`
**Upload Review Gate:** promote a submission's file from quarantine into `incoming/`.
**Synopsis**
- `wikitool upload accept <id> [--confirm TOKEN]`
**Options**
```text
Arguments:
submission_id A submission id from `upload list` [required]
Options:
--confirm <str> The token from a prior refusal, once a human has reviewed it
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool upload accept <id>
tools/wikitool upload accept <id> --confirm <token> # re-run after exit 42, once the user approved the manifest
```
**Behaviour**
- Promotes a submission's file from `mcp-upload/<id>/` into `incoming/`, deletes the quarantine directory, and appends an `accepted` event to `mcp-upload/ledger.jsonl`.
- Upload Review Gate: without a matching `--confirm`, exits 42 and prints the manifest in full plus the exact re-run line - one submission at a time. The gate check runs before anything is moved.
- The token digests id, filename, size, sha256 and submitter, so an edited or superseded manifest invalidates it.
- An occupied `incoming/<filename>` is an ordinary validation error, not the gate.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - one filesystem move, one directory delete, one ledger append; the gate check runs first, before any of them
- Counts toward the session call budget: yes
- Can reach the network: no
- Gates: upload-review - see [exit codes and gates](../exit-codes.md)
**Exit codes**
- `0` - success
- `1` - Unknown or malformed submission id, or the submission's file is missing from `mcp-upload/<id>/`
- `1` - `incoming/<filename>` already exists
- `42` - Upload Review Gate: `--confirm` is absent or does not match the manifest's current token
**See also**
- `wikitool upload show` - the manifest to review
- `wikitool raw accept` - the next step for the file in `incoming/`
- `instructions/gates.md` - the gate procedure
## `upload reject`
Delete a submission's material, keeping only its ledger trail.
**Synopsis**
- `wikitool upload reject <id> --reason "<why>"`
**Options**
```text
Arguments:
submission_id A submission id from `upload list` [required]
Options:
--reason <str> Why this submission was declined [required]
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool upload reject <id> --reason "duplicate of raw/articles/llm-wiki.md"
```
**Behaviour**
- Appends an append-only `rejected` event naming the reason and the sha256 of what was declined, then deletes the submission's material.
- The ledger write happens first, so an interruption still leaves the reason on record.
- No gate - rejecting needs no clearance, only accepting does.
- Not idempotent: a second call with the same id reports "unknown id", which confirms the first call worked.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - one ledger append, then one recursive delete; the ledger write happens first, so an interruption still leaves the reason on record
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unknown or malformed submission id, or an empty `--reason`
**See also**
- `wikitool upload show` - the manifest to review
<!-- /wikitool:reference-commands -->
docs/reference/commands/telemetry.md
+114
View File
@@ -0,0 +1,114 @@
# Telemetry
Reading and scoring the traces that sessions leave behind.
<!-- wikitool:toc -->
## Contents
- [`eval sessions`](#eval-sessions)
- [`eval score`](#eval-score)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `eval sessions`
List the sessions that have a trace under `reports/telemetry/`.
**Synopsis**
- `wikitool eval sessions [--json]`
**Options**
```text
--json Print the list as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool eval sessions
tools/wikitool eval sessions --json
```
**Behaviour**
- Lists the sessions that have a trace under `reports/telemetry/`, most recent first.
- Never fails; an empty list is a valid answer.
- Read-only and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool eval score` - scores one of them
- `EVALS.md` - how telemetry and evaluation work
## `eval score`
Score one traced session.
**Synopsis**
- `wikitool eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]`
**Options**
```text
--session <str> Session to score. Defaults to this shell's session, the
same id the budget gate uses.
--json Print the scorecard as JSON
--markdown <path> Write a markdown scorecard here, conventionally under
reports/evals/.
--save Write the scorecard to
reports/evals/<date>/<session>.{json,md}.
--fail-on-error Exit non-zero when the tree has hard errors or an invariant
was violated.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool eval score
tools/wikitool eval score --session wiki-1727330000 --save
```
**Behaviour**
- Scores one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged.
- Defaults to the current session; `--session <id>` picks another.
- `--save` writes `reports/evals/<date>/<session>.{json,md}`; `--markdown` writes the report to the named file.
- A session records nothing when telemetry is off - `WIKI_TRACE=0`, or a distributed instance with no opt-in (`wikitool doctor` says which) - so an absent trace is not necessarily a fault.
- Read-only over `kb/`, safe to retry, and exempt from the Iteration Budget Gate.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only, apart from the files `--save`/`--markdown` write
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - No trace exists for the named session
- `1` - Only with `--fail-on-error`: the tree has hard errors or an invariant was violated
**See also**
- `wikitool eval sessions` - which session ids exist
- `EVALS.md` - the scoring levels
<!-- /wikitool:reference-commands -->
docs/reference/commands/types-instructions-and-docs.md
+521
View File
@@ -0,0 +1,521 @@
# Types, instructions and docs
The page types, the instruction and skill layer, and the documentation checks and generators that keep the docs in step with the code.
<!-- wikitool:toc -->
## Contents
- [`types list`](#types-list)
- [`types describe`](#types-describe)
- [`instructions sync`](#instructions-sync)
- [`instructions verify`](#instructions-verify)
- [`instructions list`](#instructions-list)
- [`docs verify`](#docs-verify)
- [`docs toc`](#docs-toc)
- [`docs prerequisites`](#docs-prerequisites)
- [`docs contract`](#docs-contract)
- [`docs reference`](#docs-reference)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `types list`
List every type-spec under `types/`.
**Synopsis**
- `wikitool types list [--json]`
**Options**
```text
--json Print raw findings as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool types list
tools/wikitool types list --json
```
**Behaviour**
- Lists every type-spec under `types/`: name, schema path, subtype field, and description - which page types exist, without reading `types/*.md` directly.
- Never fails; read-only and safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool types describe <name>` - one type's full contract
## `types describe`
Print one type's full contract.
**Synopsis**
- `wikitool types describe <name> [--json]`
**Options**
```text
Arguments:
name Type name, e.g. 'entity' (see `types list`) [required]
Options:
--json Print raw findings as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool types describe source
tools/wikitool types describe project --json
```
**Behaviour**
- Prints one type's full contract: required and optional frontmatter fields with enums, its subtype field (if any), and its authoring body.
- Where the type-spec declares `guidance:`, the stack-owned `types/<name>.guidance.md` is composed in, so a `root: kb` type's contract reads as one answer even though it may live in two files. `--json` reports it separately as `guidance`/`guidance_path`, absent for a type with none.
- The generated table-of-contents region a long type-spec (or guidance file) carries is stripped from this output.
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unknown type name
**See also**
- `wikitool types list` - every type
- `wikitool new <type>` - scaffolds a page of the type
## `instructions sync`
Publish every `instructions/<name>/SKILL.md` into the harness skill directories.
**Synopsis**
- `wikitool instructions sync [--force]`
**Options**
```text
--force Replace a target directory whose contents differ from the source
and was not generated by sync. Without this, sync refuses instead
of deleting content it did not create.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool instructions sync
```
**Behaviour**
- Publishes every `instructions/<name>/SKILL.md` into `.agents/skills/` and `.claude/skills/` as **copies**, and deletes published skills whose source is gone.
- Both targets are gitignored, so a fresh clone runs this once.
- Re-running repairs a drifted copy: the source always wins.
- `--force` is required only to replace a target directory that is not a published skill at all (no `SKILL.md` in it).
- Each copy is idempotent, so a re-run converges even after a partial failure.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: No - one directory copy per skill per target (`.agents/skills/`, `.claude/skills/`); each copy is idempotent, so a re-run converges even after a partial failure
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - No skills found under `instructions/`
- `1` - A target directory is not a published skill (no `SKILL.md`) and `--force` was not passed
**See also**
- `instructions/bootstrap.md` - the fresh-clone procedure that runs this
- `wikitool instructions verify` - checks the copies match
## `instructions verify`
Check the instruction layer.
**Synopsis**
- `wikitool instructions verify`
**Options**
```text
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool instructions verify
```
**Behaviour**
- Flat instructions validate against `types/instruction.schema.yaml`, and each `SKILL.md` carries the frontmatter its harness reads.
- No `SKILL.md` carries a relative markdown link: `sync` copies it to a different depth than the source, so a `SKILL.md` references a target as a repo-root-relative plain path instead (`instructions/CONTRACT.md` § "A skill's outbound reference is a plain path, not a link").
- Every published copy is byte-identical to its source. Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout.
- No instruction is left that nothing references; one marked `manual: true` must instead not be linked from AGENTS.md, CLAUDE.md or a skill.
- Nothing under `instructions/dev/` is referenced from outside it; a `<!-- dist:strip-start/end -->` block is exempt (`instructions/CONTRACT.md`).
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Nothing found under `instructions/` at all, or a malformed instruction or `SKILL.md`
- `1` - A `SKILL.md` carries a relative markdown link
- `1` - A published copy drifted from its source
- `1` - An instruction nothing references, or a `manual: true` one that IS linked from AGENTS.md, CLAUDE.md or a skill and so risks running implicitly
- `1` - Something under `instructions/dev/` is referenced from outside it and outside a `dist:strip` block
**See also**
- `wikitool instructions sync` - publishes the copies
- `instructions/CONTRACT.md` - the rules this checks
## `instructions list`
List the flat instructions with their descriptions.
**Synopsis**
- `wikitool instructions list [--json]`
**Options**
```text
--json Print the listing as JSON
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool instructions list
tools/wikitool instructions list --json
```
**Behaviour**
- Lists the flat instructions with their descriptions - how the instruction layer is discovered; `search` covers `kb/` only.
- Never fails: an empty `instructions/` prints "No instructions found." Read-only and safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `wikitool search` - the same question for `kb/`
## `docs verify`
Check the docs that mirror the code.
**Synopsis**
- `wikitool docs verify`
**Options**
```text
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool docs verify
```
**Behaviour**
- Checks the docs that mirror the code. The name is about documentation parity, not the `docs/` directory - it reads that only for its layout and its generated reference pages, and never requires one.
- Commands: every command has a `cli_contract` record and is listed in `cli_contract.GROUPS`, in both directions; every command's non-hidden flags appear in its record's SYNOPSIS and vice versa; `tools/CONTRACT.md`'s generated `<!-- wikitool:commands -->` region matches what `docs contract` would write; no command's rendered `--help`/`-h` text cites an issue number.
- Collections: every directory under `kb/` has a `COLLECTION.md` and no directory outside it does; every collection declares `profile:` and a `required_by_stack:` that agrees with the stack's own list.
- Types: every type the stack lists (currently `source` and `project`) has a type-spec of that name whose schema requires the field the stack list names (`raw_files:`/`state:`); every file under `types/` declaring `type: types/type-spec.md` validates against `types/type-spec.schema.yaml`; every subtype template `types/<type>.<value>.md` (any `<value>` but `guidance`) sits beside a type-spec declaring `subtype_field:`, names a value that field's schema enum allows, and carries no frontmatter; no pre-migration `type: entity` blocks are left in the contracts.
- `kb/CONVENTIONS.md`, if it exists at all, names all three tool-owned section headings; every stage contract is present.
- The `.gitignore` canaries clear in both directions: nothing ignored under `raw/`/`kb/`, `incoming/` ignored, everything ignored under `reports/` and the published skill directories.
- No `.md`/`.template` file `dist export` would ship cites an issue number. A `<!-- dist:strip-start/end -->` region is exempt: the check reads the export plan's text, from which it is already gone.
- Every reference file `docs toc` covers carries the current table-of-contents region for its own headings - missing and stale are one check.
- Every relative markdown link in one of those reference files resolves to an existing file. A target's `#anchor` suffix is stripped first, and code fences and inline code spans are masked before scanning, so link syntax shown as an example is not mistaken for a real reference.
- Every file under `docs/` sits in one of its areas - `tutorials/`, `how-to/`, `reference/`, `explanation/` or `dev/` - never directly in `docs/` or in another directory.
- If `docs/` exists: every generated region of the human command reference under `docs/reference/` matches what `docs reference` would write, every `cli_contract.GROUPS` entry has its page under `docs/reference/commands/`, and no page there belongs to no group.
- `INSTALL.md` carries one generated `<!-- wikitool:prerequisites -->` region per platform value of `tools/prerequisites.txt` (`prerequisites-<platform>` for a platform-specific one), each current; and the `<!-- setup-question: <key> -->` markers in `instructions/setup-instance.md` and `INSTALL.md` name the same set of keys, so a question the agent asks is never one the human guide leaves out, nor the reverse.
- `VERSION` parses and the newest versioned `CHANGES.md` entry names it. An entry that crosses the compatibility boundary from the last release carries `**Breaking Change:**`, has a migration document or `**Migration:** none required`, and its `**Migration:**` line names every required migration document targeting it.
- The newest versioned `CHANGES.md` entry is at most 32 000 bytes (UTF-8) as `version notes` prints it, and each of its `###` sections is exactly one paragraph of at most 1 000 bytes, heading line excluded. Older entries and a changelog with no versioned entry are not checked.
- Read-only.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - A command, contract, or type-form mismatch
- `1` - `VERSION` and the newest `CHANGES.md` entry disagree, or a boundary-crossing entry lacks its breaking line, its migration, or the migration document's path
- `1` - The newest `CHANGES.md` entry is over 32 000 bytes, or one of its `###` sections has more than one paragraph or a paragraph over 1 000 bytes
- `1` - The `<!-- wikitool:commands -->` region of `tools/CONTRACT.md` is stale
- `1` - A type-spec's own frontmatter fails its schema
- `1` - A subtype template `types/<type>.<value>.md` has no type-spec with a `subtype_field:` beside it, names a value outside that field's enum, or carries frontmatter
- `1` - A shipped `.md`/`.template` cites an issue number
- `1` - A region of the human command reference under `docs/reference/` is stale
- `1` - A page of the human command reference is missing or lacks its marker pair, or a page under `docs/reference/commands/` belongs to no group
- `1` - A reference file's table-of-contents region is missing or stale
- `1` - A file sits directly in `docs/`, or in a directory there that is not one of its five areas
- `1` - A reference file's relative markdown link does not resolve to an existing file
- `1` - An `INSTALL.md` prerequisites region is stale
- `1` - An `INSTALL.md` prerequisites region is missing, or names a platform no tool has
- `1` - A setup question is marked in one of `instructions/setup-instance.md` and `INSTALL.md` but not the other
**See also**
- `wikitool docs toc` - regenerates tables of contents
- `wikitool docs contract` - regenerates the commands region
- `wikitool docs reference` - regenerates the human command reference
- `wikitool docs prerequisites` - regenerates `INSTALL.md`'s prerequisites lists
- `wikitool instructions verify` - the same kind of check for `instructions/`
## `docs toc`
Create, refresh or remove the generated table-of-contents region.
**Synopsis**
- `wikitool docs toc [--apply]`
**Options**
```text
--apply Write changes; default is dry-run (preview only)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool docs toc
tools/wikitool docs toc --apply
```
**Behaviour**
- Creates, refreshes or removes the generated table-of-contents region on every reference file over 100 lines: `AGENTS.md`, every stage contract, `kb/CONVENTIONS.md`, every `kb/*/COLLECTION.md`, every flat `instructions/**.md` file, every `types/*.md` type-spec, and every `docs/` page - each together with the `<name>.template` it ships as, where one exists.
- The scope is computed from those categories rather than listed, so a file added later is in scope without a code change.
- Out of scope: every `SKILL.md`, the human docs (`README.md`, `CHANGES.md`, `EVALS.md`, `INSTALL.md`, `tools/README.md`), and every subtype template `types/<type>.<value>.md` - `new` copies it into a page, so it never carries a region whatever its length.
- Dry-run by default (prints which files would change); `--apply` writes. Idempotent: a re-run after an interruption converges rather than doubling a region.
- If `docs verify` still reports a stale region after `--apply`, the file's `##` headings changed in between; run it again.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: `--apply` rewrites each named file in place, one at a time and idempotently, so a re-run after an interruption converges rather than doubling a region; the dry-run form is read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `0` - Never fails on content: a file with no `##` heading, or one at or under the threshold, is simply left without a region
**See also**
- `wikitool docs verify` - checks every region is current
## `docs prerequisites`
Regenerate `INSTALL.md`'s prerequisites lists from `tools/prerequisites.txt`.
**Synopsis**
- `wikitool docs prerequisites [--apply]`
**Options**
```text
--apply Write changes; default is dry-run (preview only)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool docs prerequisites
tools/wikitool docs prerequisites --apply
```
**Behaviour**
- Rewrites each `<!-- wikitool:prerequisites -->` region in `INSTALL.md` (tools every platform needs) and `<!-- wikitool:prerequisites-<platform> -->` region (tools only that platform needs) from the manifest: one list item per tool, its label and minimum version. The manifest's reason field stays out - it is English prose, and the region sits in a document that need not be.
- Never places a region: where a list belongs in the human guide is that guide's own decision. A region the manifest calls for but the file lacks is an error naming the marker pair to add.
- Dry-run by default (says whether the file would change); `--apply` writes.
- `docs verify` checks the result stays current.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - every region is rewritten in one file write
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `INSTALL.md` is missing, or lacks a region the manifest calls for
**See also**
- `wikitool docs verify` - checks every region is current
## `docs contract`
Regenerate `tools/CONTRACT.md`'s `<!-- wikitool:commands -->` region.
**Synopsis**
- `wikitool docs contract [--apply]`
**Options**
```text
--apply Write changes; default is dry-run (preview only)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool docs contract
tools/wikitool docs contract --apply
```
**Behaviour**
- Rebuilds the region from every `cli_contract` record: the index (one line per command, `GROUPS` order) followed by each `###` group's commands as `#### <path>` man-page-shaped sections.
- Dry-run by default (says whether the file would change); `--apply` writes.
- `docs verify` checks the result stays current.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: Yes - the whole region is rewritten in one file write
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `tools/CONTRACT.md` is missing
**See also**
- `tools/README.md` § Adding a command - how a command gets its record
- `wikitool docs reference` - the same records, rendered for a person under `docs/reference/`
- `wikitool docs verify` - checks the region is current
## `docs reference`
Regenerate the human command reference under `docs/reference/`.
**Synopsis**
- `wikitool docs reference [--apply]`
**Options**
```text
--apply Write changes; default is dry-run (preview only)
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool docs reference
tools/wikitool docs reference --apply
```
**Behaviour**
- Rewrites the generated region of every page of the human command reference from the `cli_contract` records: one `docs/reference/commands/<group>.md` per `cli_contract.GROUPS` entry (`<!-- wikitool:reference-commands -->`), the overview in `docs/reference/commands/README.md` (`<!-- wikitool:reference-commands-index -->`), and the gates and exit-42 causes in `docs/reference/exit-codes.md` (`<!-- wikitool:reference-exit-codes -->`).
- Renders each command's summary, synopsis, options, examples, notes, properties in words, exit codes with their causes and see-also - but not the ON FAILURE reactions or NEVER, which address an agent; `tools/CONTRACT.md` keeps those.
- Refreshes the table of contents of each page it writes in the same step.
- Never creates a page or places a region: a page's title and one-sentence head are written by hand. A group's page is named after the group - lower case, every run of other characters one hyphen.
- Dry-run by default (lists the pages that would change); `--apply` writes.
- `docs verify` checks the result stays current.
**Properties**
- Changes files: yes
- Idempotent: yes
- Atomic: `--apply` rewrites each page in place, one at a time and idempotently, so a re-run after an interruption converges; the dry-run form is read-only
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - A reference page is missing, or lacks its region's marker pair
**See also**
- `wikitool docs contract` - the same records, rendered for an agent into `tools/CONTRACT.md`
- `wikitool docs verify` - checks every reference region is current
- `tools/README.md` § Adding a command - how a command gets its record
<!-- /wikitool:reference-commands -->
docs/reference/commands/workshop-runs-and-session-budget.md
+216
View File
@@ -0,0 +1,216 @@
# Workshop runs and session budget
Opening and closing a workshop run under `work/`, and inspecting the per-session call budget.
<!-- wikitool:toc -->
## Contents
- [`work new`](#work-new)
- [`work close`](#work-close)
- [`budget status`](#budget-status)
- [`budget reset`](#budget-reset)
<!-- /wikitool:toc -->
<!-- wikitool:reference-commands -->
## `work new`
Scaffold `work/<runkey>/` for one workshop run.
**Synopsis**
- `wikitool work new (--input <raw path> | --key <run key>) [--again] [--dry-run]`
**Options**
```text
--input <str> Path to the raw source tree or file this run covers, e.g.
raw/documents/handbook
--key <str> Explicit run key for a run with no raw input (a migration, a
sweep across kb/). Mutually exclusive with --input; may not
start with 'ingest-'.
--again This is a deliberate re-ingest of a tree already processed
before: append today's date to the run key instead of refusing
the collision.
--dry-run Show what would be created, write nothing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool work new --input raw/2026/09/handbuch
tools/wikitool work new --key migrate-7.0.0
tools/wikitool work new --input raw/2026/09/handbuch --again
```
**Behaviour**
- Creates `work/<runkey>/` with the required `README.md` and `plan.md`.
- `--input` derives the run key from the path below `raw/` (an ingest); `--key` names it outright for a run with no raw input - a migration or a sweep across `kb/` - and may not start with `ingest-`, which stays reserved for derived keys. Exactly one of the two.
- Refuses a collision instead of suffixing it.
- `--again` opens a dated second pass (`<runkey>-<date>`) over a tree that has itself changed.
- `--dry-run` reports the run key and files without writing.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Yes - one directory with two files
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Neither or both of `--input`/`--key` given, or a `--key` that is empty or starts with `ingest-`
- `1` - `--input` is outside `raw/`, does not exist, or is `raw/` itself (an empty run key)
- `1` - The workshop already exists
**See also**
- `work/CONTRACT.md` - run keys, required files, how a run closes
- `wikitool work close` - deletes the run when it is done
- `instructions/ingest-large-tree.md` - the ingest that opens a run
## `work close`
Delete a finished workshop.
**Synopsis**
- `wikitool work close --run-key <name> [--yes] [--dry-run]`
**Options**
```text
--run-key <str> The workshop directory name [required]
-y, --yes Confirm deletion. Required: closing discards the only copy of
the run's working notes, so the durable conclusions must
already be in kb/.
--dry-run Show what would be deleted, delete nothing
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool work close --run-key ingest-2026-09-handbuch --dry-run
tools/wikitool work close --run-key ingest-2026-09-handbuch --yes
```
**Behaviour**
- Deletes `work/<run-key>/` recursively.
- Without `--yes` it lists what would be lost and refuses: nothing in a workshop is recoverable from the rest of the repo, so the durable conclusions must already be in `kb/`.
- `--dry-run` lists the files without deleting.
- Not idempotent: once deleted, the run key is unknown.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: No - a recursive delete
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - Unknown run key
- `1` - `--yes` was not passed - the output lists what would be lost
**See also**
- `work/CONTRACT.md` - how a run closes
- `wikitool work new` - opens a run
## `budget status`
Show the current session's `wikitool` call count and recent command history.
**Synopsis**
- `wikitool budget status`
**Options**
```text
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool budget status
```
**Behaviour**
- Shows the current session's `wikitool` call count and its recent command history.
- Never counted against the budget; never fails; safe to retry freely.
**Properties**
- Changes files: no
- Idempotent: yes
- Atomic: Read-only
- Counts toward the session call budget: no
- Can reach the network: no
**Exit codes**
- `0` - success
**See also**
- `instructions/session-setup.md` - scoping the budget to a task
- `instructions/gates.md` - what to do when the Iteration Budget Gate refuses
## `budget reset`
Clear the current session's (or every session's) iteration budget state.
**Synopsis**
- `wikitool budget reset --yes [--all]`
**Options**
```text
--all Clear every session's budget, not just the current one.
-y, --yes Confirm clearing the budget. Only pass this after a human has
approved continuing the session - never set it automatically to
work around the gate.
-h, --help Show this message and exit.
```
**Examples**
```bash
tools/wikitool budget reset --yes # only after the user approved it
```
**Behaviour**
- Clears the current session's iteration budget state; `--all` deletes every session's.
- Requires `--yes`: clearing the counter is itself a way around the Iteration Budget Gate, so it needs the same explicit human approval.
- Counted against the budget like any other call.
**Properties**
- Changes files: yes
- Idempotent: no
- Atomic: Read/rewrite of one JSON file (or its deletion, with `--all`)
- Counts toward the session call budget: yes
- Can reach the network: no
**Exit codes**
- `0` - success
- `1` - `--yes` not passed
**See also**
- `wikitool budget status` - the current count
- `instructions/gates.md` - why `budget reset` is not the escape hatch
<!-- /wikitool:reference-commands -->
docs/reference/exit-codes.md
+45
View File
@@ -0,0 +1,45 @@
# Exit codes and gates
Which `wikitool` commands can stop and wait for a person - exit 42 - and for which gate, generated
from the command records.
<!-- wikitool:reference-exit-codes -->
What each exit code means, and how a caller reacts to it, is [AGENTS.md § Tool error contract](../../AGENTS.md#tool-error-contract). What a gate is and why it is enforced in code is [AGENTS.md § Gates](../../AGENTS.md#gates). This page lists which commands declare a gate, and every cause for which a command exits 42.
## Gates by command
### guideline-push
- [`export guidelines`](commands/guideline-export.md#export-guidelines)
### human-intervention-required (`new project` only)
- [`new`](commands/pages.md#new)
### mass-update
- [`publish`](commands/git.md#publish)
### publish-remote
- [`publish`](commands/git.md#publish)
### rebase-review
- [`sync`](commands/git.md#sync)
- [`publish`](commands/git.md#publish)
### upload-review
- [`upload accept`](commands/raw-material-and-uploads.md#upload-accept)
## Exit 42 causes
- [`new`](commands/pages.md#new): new project: The provider cannot create the project itself (Super Productivity's `access: "api"`); the output says what a human has to create
- [`upload accept`](commands/raw-material-and-uploads.md#upload-accept): Upload Review Gate: `--confirm` is absent or does not match the manifest's current token
- [`sync`](commands/git.md#sync): Rebase-review gate: `<remote>/<branch>` moved and both sides changed the same file, generated files aside; the output lists the upstream commits, the overlapping files and their diff
- [`publish`](commands/git.md#publish): Mass-Update Gate: `--threshold` (default 10) or more counted files would be committed, or the `--confirm` token does not match this changeset
- [`publish`](commands/git.md#publish): Rebase-review gate: `<remote>/<branch>` moved and both sides changed the same file; the output lists the upstream commits, the overlapping files and their diff
- [`publish`](commands/git.md#publish): Publish-Remote Gate: `.wikitool-remotes.json` exists and the push URL of `--remote` is not listed in it, or `--remote` resolves to no push URL
- [`export guidelines`](commands/guideline-export.md#export-guidelines): Guideline Push Gate: `--push` without `--confirm`, or with a token that does not match the current set of targets
<!-- /wikitool:reference-exit-codes -->
instructions/dev/doc-pull-through.md
+1 -1
View File
@@ -33,7 +33,7 @@ touched; a row that does not apply needs no action.
| Touched surface | Document(s) that make a claim about it |
|---|---|
| A `wikitool` command's behaviour, flags, or interface | Its `cli_contract.CommandRecord` (name, synopsis, properties, exit status, retry policy - `tools/chemenu/cli_contract.py`), then `wikitool docs contract --apply` to regenerate its copy in [tools/CONTRACT.md](../../tools/CONTRACT.md) |
| A `wikitool` command's behaviour, flags, or interface | Its `cli_contract.CommandRecord` (name, synopsis, properties, exit status, retry policy - `tools/chemenu/cli_contract.py`), then `wikitool docs contract --apply` to regenerate its copy in [tools/CONTRACT.md](../../tools/CONTRACT.md) and `wikitool docs reference --apply` for the human reference under `docs/reference/` (a new group in `cli_contract.GROUPS` first needs its page there, with a title, a one-sentence head and the marker pair) |
| A stage's authoring rules (`raw/`, `kb/`, `types/`, `reports/`, `work/`, `tools/`, `instructions/`) | The touched `<stage>/CONTRACT.md` |
| A rule, gate, or invariant `AGENTS.md` itself states | The relevant `AGENTS.md` section (Invariants, Gates, File naming, Routing, ...) |
| A workflow, stage, or command a human operates by hand | Whichever of `README.md`, `EVALS.md`, `tools/README.md`, `INSTALL.md`, `INSTALL-MCP.md` or the pages under `docs/tutorials/`, `docs/how-to/` and `docs/reference/` names it - AGENTS.md § File naming says which document is for which reader. A new page goes into the area its kind of documentation belongs to; that area's `README.md` says which |
tools/CONTRACT.md
+65 -3
View File
@@ -11,7 +11,7 @@ not here.
**Look up the command you need; do not read this file end to end.** Every
command carries one data record - name, synopsis, properties, exit status, and
so on - attached to its function in code (`tools/chemenu/cli_contract.py`).
Three views render from that one source:
Three views for an agent render from that one source:
```bash
tools/wikitool raw accept -h # the full record, for one command
@@ -24,6 +24,11 @@ The § Commands section below is `wikitool docs contract`'s output: an index
as `#### <path>` records, in the same order `wikitool -h` prints. Reading the
file end to end is for changing the CLI itself.
A fourth view is for a person: `wikitool docs reference` renders the same
records into one page per group under
[`docs/reference/commands/`](../docs/reference/commands/README.md), leaving out
ON FAILURE and NEVER, which address an agent.
<!-- wikitool:toc -->
## Contents
@@ -140,6 +145,7 @@ docs verify read idempotent budget:counted exit:0,1
docs toc write idempotent budget:counted exit:0 Create, refresh or remove the generated table-of-contents region.
docs prerequisites write idempotent budget:counted exit:0,1 Regenerate `INSTALL.md`'s prerequisites lists from `tools/prerequisites.txt`.
docs contract write idempotent budget:counted exit:0,1 Regenerate `tools/CONTRACT.md`'s `<!-- wikitool:commands -->` region.
docs reference write idempotent budget:counted exit:0,1 Regenerate the human command reference under `docs/reference/`.
eval sessions read idempotent budget:exempt exit:0 List the sessions that have a trace under `reports/telemetry/`.
eval score read idempotent budget:exempt exit:0,1 Score one traced session.
dist export write idempotent budget:counted exit:0,1 Write a contentless, distributable copy of this repo's machinery.
@@ -2510,6 +2516,8 @@ Check the docs that mirror the code.
- 1 A type-spec's own frontmatter fails its schema
- 1 A subtype template `types/<type>.<value>.md` has no type-spec with a `subtype_field:` beside it, names a value outside that field's enum, or carries frontmatter
- 1 A shipped `.md`/`.template` cites an issue number
- 1 A region of the human command reference under `docs/reference/` is stale
- 1 A page of the human command reference is missing or lacks its marker pair, or a page under `docs/reference/commands/` belongs to no group
- 1 A reference file's table-of-contents region is missing or stale
- 1 A file sits directly in `docs/`, or in a directory there that is not one of its five areas
- 1 A reference file's relative markdown link does not resolve to an existing file
@@ -2526,6 +2534,8 @@ Check the docs that mirror the code.
- A type-spec's own frontmatter fails its schema -> Fix the field, or add a matching line to `types/type-spec.schema.yaml` if the field is legitimately new
- A subtype template `types/<type>.<value>.md` has no type-spec with a `subtype_field:` beside it, names a value outside that field's enum, or carries frontmatter -> Rename the file to the type and value it was meant for, delete it, or remove its frontmatter block
- A shipped `.md`/`.template` cites an issue number -> Say what was decided instead of pointing at where, or move the pointer behind a `<!-- dist:strip-start/end -->` block
- A region of the human command reference under `docs/reference/` is stale -> Run `docs reference --apply`, then re-run
- A page of the human command reference is missing or lacks its marker pair, or a page under `docs/reference/commands/` belongs to no group -> Create the page with a title, a one-sentence head and the marker pair the message names (or add the pair), then run `docs reference --apply`; `git mv` or `git rm` a page left over from a renamed or removed group
- A reference file's table-of-contents region is missing or stale -> Run `docs toc --apply`, then re-run
- A file sits directly in `docs/`, or in a directory there that is not one of its five areas -> Move it (`git mv`) into the area its kind of documentation belongs to, and fix the links that pointed at the old path
- A reference file's relative markdown link does not resolve to an existing file -> Fix the `../` count or the target's name
@@ -2535,11 +2545,11 @@ Check the docs that mirror the code.
**NEVER**
- Never hand-write a table-of-contents region or the commands region - regenerate it.
- Never hand-write a table-of-contents region, the commands region or a reference region - regenerate it.
**NOTES**
- Checks the docs that mirror the code. The name is about documentation parity, not the `docs/` directory - it reads that only for its layout, and never requires one.
- Checks the docs that mirror the code. The name is about documentation parity, not the `docs/` directory - it reads that only for its layout and its generated reference pages, and never requires one.
- Commands: every command has a `cli_contract` record and is listed in `cli_contract.GROUPS`, in both directions; every command's non-hidden flags appear in its record's SYNOPSIS and vice versa; `tools/CONTRACT.md`'s generated `<!-- wikitool:commands -->` region matches what `docs contract` would write; no command's rendered `--help`/`-h` text cites an issue number.
- Collections: every directory under `kb/` has a `COLLECTION.md` and no directory outside it does; every collection declares `profile:` and a `required_by_stack:` that agrees with the stack's own list.
- Types: every type the stack lists (currently `source` and `project`) has a type-spec of that name whose schema requires the field the stack list names (`raw_files:`/`state:`); every file under `types/` declaring `type: types/type-spec.md` validates against `types/type-spec.schema.yaml`; every subtype template `types/<type>.<value>.md` (any `<value>` but `guidance`) sits beside a type-spec declaring `subtype_field:`, names a value that field's schema enum allows, and carries no frontmatter; no pre-migration `type: entity` blocks are left in the contracts.
@@ -2549,6 +2559,7 @@ Check the docs that mirror the code.
- Every reference file `docs toc` covers carries the current table-of-contents region for its own headings - missing and stale are one check.
- Every relative markdown link in one of those reference files resolves to an existing file. A target's `#anchor` suffix is stripped first, and code fences and inline code spans are masked before scanning, so link syntax shown as an example is not mistaken for a real reference.
- Every file under `docs/` sits in one of its areas - `tutorials/`, `how-to/`, `reference/`, `explanation/` or `dev/` - never directly in `docs/` or in another directory.
- If `docs/` exists: every generated region of the human command reference under `docs/reference/` matches what `docs reference` would write, every `cli_contract.GROUPS` entry has its page under `docs/reference/commands/`, and no page there belongs to no group.
- `INSTALL.md` carries one generated `<!-- wikitool:prerequisites -->` region per platform value of `tools/prerequisites.txt` (`prerequisites-<platform>` for a platform-specific one), each current; and the `<!-- setup-question: <key> -->` markers in `instructions/setup-instance.md` and `INSTALL.md` name the same set of keys, so a question the agent asks is never one the human guide leaves out, nor the reverse.
- `VERSION` parses and the newest versioned `CHANGES.md` entry names it. An entry that crosses the compatibility boundary from the last release carries `**Breaking Change:**`, has a migration document or `**Migration:** none required`, and its `**Migration:**` line names every required migration document targeting it.
- The newest versioned `CHANGES.md` entry is at most 32 000 bytes (UTF-8) as `version notes` prints it, and each of its `###` sections is exactly one paragraph of at most 1 000 bytes, heading line excluded. Older entries and a changelog with no versioned entry are not checked.
@@ -2558,6 +2569,7 @@ Check the docs that mirror the code.
- `wikitool docs toc` - regenerates tables of contents
- `wikitool docs contract` - regenerates the commands region
- `wikitool docs reference` - regenerates the human command reference
- `wikitool docs prerequisites` - regenerates `INSTALL.md`'s prerequisites lists
- `wikitool instructions verify` - the same kind of check for `instructions/`
@@ -2691,8 +2703,58 @@ Regenerate `tools/CONTRACT.md`'s `<!-- wikitool:commands -->` region.
**SEE ALSO**
- `tools/README.md` § Adding a command - how a command gets its record
- `wikitool docs reference` - the same records, rendered for a person under `docs/reference/`
- `wikitool docs verify` - checks the region is current
#### `docs reference`
Regenerate the human command reference under `docs/reference/`.
**SYNOPSIS**
- `wikitool docs reference [--apply]`
**PROPERTIES**
- effect: write
- idempotent: yes
- atomic: `--apply` rewrites each page in place, one at a time and idempotently, so a re-run after an interruption converges; the dry-run form is read-only
- budget: counted
- network: no
**EXAMPLES**
- `tools/wikitool docs reference`
- `tools/wikitool docs reference --apply`
**EXIT STATUS**
- 0 success
- 1 A reference page is missing, or lacks its region's marker pair
**ON FAILURE**
- A reference page is missing, or lacks its region's marker pair -> Not transient - create the page the message names with a title, a one-sentence head and the marker pair, or add the marker pair below its head, then retry
**NEVER**
- Never hand-edit a reference region - change the record in code and re-run this.
**NOTES**
- Rewrites the generated region of every page of the human command reference from the `cli_contract` records: one `docs/reference/commands/<group>.md` per `cli_contract.GROUPS` entry (`<!-- wikitool:reference-commands -->`), the overview in `docs/reference/commands/README.md` (`<!-- wikitool:reference-commands-index -->`), and the gates and exit-42 causes in `docs/reference/exit-codes.md` (`<!-- wikitool:reference-exit-codes -->`).
- Renders each command's summary, synopsis, options, examples, notes, properties in words, exit codes with their causes and see-also - but not the ON FAILURE reactions or NEVER, which address an agent; `tools/CONTRACT.md` keeps those.
- Refreshes the table of contents of each page it writes in the same step.
- Never creates a page or places a region: a page's title and one-sentence head are written by hand. A group's page is named after the group - lower case, every run of other characters one hyphen.
- Dry-run by default (lists the pages that would change); `--apply` writes.
- `docs verify` checks the result stays current.
**SEE ALSO**
- `wikitool docs contract` - the same records, rendered for an agent into `tools/CONTRACT.md`
- `wikitool docs verify` - checks every reference region is current
- `tools/README.md` § Adding a command - how a command gets its record
### Telemetry
#### `eval sessions`
tools/README.md
+6 -1
View File
@@ -164,7 +164,12 @@ bound at import time - `KB_DIR` and friends follow whatever `ROOT` currently is.
a comment next to the code). `docs verify` fails in both directions - a command with
no record and a `GROUPS` entry naming no real command are equally reported -
and also checks that every non-hidden flag appears in the record's SYNOPSIS.
Then regenerate the copy: `wikitool docs contract --apply`. Write the
Then regenerate both copies: `wikitool docs contract --apply` for
CONTRACT.md and `wikitool docs reference --apply` for the human reference
under `docs/reference/commands/`. A new group needs its page there first -
`docs/reference/commands/<group>.md` with a title, a one-sentence head and
the `<!-- wikitool:reference-commands -->` marker pair; the command never
creates one. Write the
record's prose without citing an issue number: `docs verify` also refuses
one in a command's rendered `--help` text (a docstring above its `\f`
marker, or an option's `help=`) and in any `.md`/`.template` `dist export`
tools/chemenu/cli.py
+24 -2
View File
@@ -193,7 +193,7 @@ app.command("sync")(git_publish.sync_command)
app.command("doctor")(doctor.doctor_command)
def _render_options_text(command, ctx) -> str:
def render_options_text(command, ctx) -> str:
"""Click's own Arguments/Options sections, plain-formatted, for splicing
into a `cli_contract` record's OPTIONS section. A fixed width (not the
real terminal's) is what keeps `wikitool <cmd> -h` byte-identical with
@@ -210,6 +210,28 @@ def _render_options_text(command, ctx) -> str:
return text
def options_texts(paths) -> dict[str, str]:
"""`render_options_text` for each command path in `paths`, outside a
help request - for the generated reference under `docs/reference/`.
Each context is built down the real chain from the root with
`resilient_parsing`, as Click does for `--help`, so the result carries the
root's `-h` alias and matches the OPTIONS section of `wikitool <path> -h`
byte for byte."""
import typer.main
root = typer.main.get_command(app)
texts: dict[str, str] = {}
for path in paths:
ctx = root.make_context("wikitool", [], resilient_parsing=True)
command = root
for name in path.split():
command = command.commands[name]
ctx = command.make_context(name, [], parent=ctx, resilient_parsing=True)
texts[path] = render_options_text(command, ctx)
return texts
def _render_root_help() -> str:
"""`wikitool -h`/`--help`/no-args: usage, the full index, and where the
per-command record lives - never Click's default subcommand listing,
@@ -258,7 +280,7 @@ try:
_original_format_help(self, ctx, formatter)
return
formatter.write(
cli_contract.render_text(record, options_text=_render_options_text(self, ctx))
cli_contract.render_text(record, options_text=render_options_text(self, ctx))
)
_typer_click_core.Command.format_help = _contract_format_help
tools/chemenu/cli_contract.py
+19 -12
View File
@@ -1,7 +1,9 @@
"""One data record per `wikitool` command - the single source three views are
"""One data record per `wikitool` command - the single source four views are
rendered from: `wikitool <cmd> -h` (the full record, plain text), the index
line (`wikitool -h` and the top of `tools/CONTRACT.md`), and the generated
`<!-- wikitool:commands -->` region of `tools/CONTRACT.md` itself.
line (`wikitool -h` and the top of `tools/CONTRACT.md`), the generated
`<!-- wikitool:commands -->` region of `tools/CONTRACT.md` itself, and the
human command reference under `docs/reference/` (`render_reference_*`), which
leaves out what is addressed to an agent - the ON FAILURE reactions and NEVER.
A record is attached to its command function, in that function's own module,
via the `@record(...)` decorator - never centralised, so the contract sits
@@ -274,7 +276,7 @@ GROUPS: tuple[tuple[str, tuple[str, ...]], ...] = (
("Types, instructions and docs", (
"types list", "types describe",
"instructions sync", "instructions verify", "instructions list",
"docs verify", "docs toc", "docs prerequisites", "docs contract",
"docs verify", "docs toc", "docs prerequisites", "docs contract", "docs reference",
)),
("Telemetry", (
"eval sessions", "eval score",
@@ -359,17 +361,22 @@ def render_properties_lines(props: Properties) -> list[str]:
return lines
def render_exit_status_lines(rec: CommandRecord) -> list[str]:
"""`0 success` first, then one line per `Failure` in code order (stable
within a code). A command with gates but no explicit exit-42 cause gets
one generic 42 line naming them."""
lines = ["0 success"]
def exit_status_entries(rec: CommandRecord) -> list[tuple[int, str]]:
"""`(0, "success")` first, then one `(code, cause)` per `Failure` in code
order (stable within a code). A command with gates but no explicit exit-42
cause gets one generic 42 entry naming them."""
entries = [(0, "success")]
for failure in sorted(rec.failures, key=lambda f: f.code):
lines.append(f"{str(failure.code).ljust(4)} {_labelled(failure, failure.cause)}")
entries.append((failure.code, _labelled(failure, failure.cause)))
if rec.properties.gates and not any(f.code == 42 for f in rec.failures):
gate_list = ", ".join(rec.properties.gates)
lines.append(f"42 needs clearance - {gate_list} (see AGENTS.md § Gates)")
return lines
entries.append((42, f"needs clearance - {gate_list} (see AGENTS.md § Gates)"))
return entries
def render_exit_status_lines(rec: CommandRecord) -> list[str]:
"""EXIT STATUS as `<code> <cause>` lines - see `exit_status_entries`."""
return [f"{str(code).ljust(4)} {cause}" for code, cause in exit_status_entries(rec)]
def render_on_failure_lines(rec: CommandRecord) -> list[str]:
tools/chemenu/commands/docs_verify.py
+158 -3
View File
@@ -76,7 +76,7 @@ from typing import Optional
import typer
from chemenu import blocks, cli_contract, config, conventions, install_doc, kb_collections, markdown_code, toc, toolpaths, version as version_mod
from chemenu import blocks, cli_contract, config, conventions, install_doc, kb_collections, markdown_code, reference_doc, toc, toolpaths, version as version_mod
from chemenu.commands import dist_cmd
from chemenu.commands._util import fail, rel_path, success
@@ -336,6 +336,47 @@ def check_commands_region() -> list[str]:
return []
def check_reference_regions() -> list[str]:
"""Every generated region of the human command reference under
`docs/reference/` matches what `docs reference` would write, every
`GROUPS` entry has its page, and no page is left over from a group that
is gone.
A tree without `docs/` passes, as for `check_docs_layout`: `docs verify`
says what a present `docs/` must hold, not that one must exist. With
`docs/` present, every page is required - a deleted one is drift too.
"""
if not (config.ROOT / "docs").is_dir():
return []
issues = []
for region in reference_doc.regions():
path = config.ROOT / region.path
if not path.is_file():
issues.append(
f"{region.path} is missing - create it with a title, a one-sentence head and the "
f"marker pair `{blocks.open_marker(region.name)}` / "
f"`{blocks.close_marker(region.name)}`, then run `wikitool docs reference --apply`"
)
continue
existing = blocks.find(path.read_text(encoding="utf-8"), region.name)
if existing is None:
issues.append(
f"{region.path} has no `{blocks.open_marker(region.name)}` region - add the marker "
"pair below the page's head, then run `wikitool docs reference --apply`"
)
elif existing.strip("\n") != region.render().strip("\n"):
issues.append(
f"{region.path}'s `{blocks.open_marker(region.name)}` region is stale - run "
"`wikitool docs reference --apply`"
)
issues += [
f"{rel_path(page)} belongs to no group in cli_contract.GROUPS - `git mv` it to the page "
"of the group it was renamed to, or `git rm` it"
for page in reference_doc.orphan_pages()
]
return issues
# A `--flag` or `-f` token in a SYNOPSIS usage string. Excludes anything that
# is not immediately preceded/followed by a word character or hyphen, so
# `--set field=value` matches only `--set`, never `field` or `value`.
@@ -831,6 +872,10 @@ def command_table_free_readmes() -> list[Path]:
Built at call time rather than at import, so a test can point ROOT_README at
a fixture.
`docs/reference/commands/README.md` is deliberately not one of them: its
overview of every command is a generated region `check_reference_regions`
holds to the records - the checked kind of copy, not the drifting kind.
"""
return [ROOT_README, *(config.ROOT / relative for relative in STAGE_READMES)]
@@ -1208,7 +1253,8 @@ def check_breaking_change_for_boundary() -> list[str]:
),
notes=(
"Checks the docs that mirror the code. The name is about documentation parity, not the "
"`docs/` directory - it reads that only for its layout, and never requires one.",
"`docs/` directory - it reads that only for its layout and its generated reference "
"pages, and never requires one.",
"Commands: every command has a `cli_contract` record and is listed in "
"`cli_contract.GROUPS`, in both directions; every command's non-hidden flags appear in "
"its record's SYNOPSIS and vice versa; `tools/CONTRACT.md`'s generated "
@@ -1242,6 +1288,10 @@ def check_breaking_change_for_boundary() -> list[str]:
"Every file under `docs/` sits in one of its areas - `tutorials/`, `how-to/`, "
"`reference/`, `explanation/` or `dev/` - never directly in `docs/` or in another "
"directory.",
"If `docs/` exists: every generated region of the human command reference under "
"`docs/reference/` matches what `docs reference` would write, every `cli_contract.GROUPS` "
"entry has its page under `docs/reference/commands/`, and no page there belongs to no "
"group.",
"`INSTALL.md` carries one generated `<!-- wikitool:prerequisites -->` region per "
"platform value of `tools/prerequisites.txt` (`prerequisites-<platform>` for a "
"platform-specific one), each current; and the `<!-- setup-question: <key> -->` markers "
@@ -1294,6 +1344,17 @@ def check_breaking_change_for_boundary() -> list[str]:
reaction="Say what was decided instead of pointing at where, or move the pointer "
"behind a `<!-- dist:strip-start/end -->` block",
),
cli_contract.Failure(
cause="A region of the human command reference under `docs/reference/` is stale",
reaction="Run `docs reference --apply`, then re-run",
),
cli_contract.Failure(
cause="A page of the human command reference is missing or lacks its marker pair, or "
"a page under `docs/reference/commands/` belongs to no group",
reaction="Create the page with a title, a one-sentence head and the marker pair the "
"message names (or add the pair), then run `docs reference --apply`; `git mv` or "
"`git rm` a page left over from a renamed or removed group",
),
cli_contract.Failure(
cause="A reference file's table-of-contents region is missing or stale",
reaction="Run `docs toc --apply`, then re-run",
@@ -1330,11 +1391,13 @@ def check_breaking_change_for_boundary() -> list[str]:
"tools/wikitool docs verify",
),
never=(
"Never hand-write a table-of-contents region or the commands region - regenerate it.",
"Never hand-write a table-of-contents region, the commands region or a reference region "
"- regenerate it.",
),
see_also=(
"`wikitool docs toc` - regenerates tables of contents",
"`wikitool docs contract` - regenerates the commands region",
"`wikitool docs reference` - regenerates the human command reference",
"`wikitool docs prerequisites` - regenerates `INSTALL.md`'s prerequisites lists",
"`wikitool instructions verify` - the same kind of check for `instructions/`",
),
@@ -1359,6 +1422,7 @@ def verify():
+ check_changelog_budget()
+ check_no_issue_references()
+ check_docs_layout()
+ check_reference_regions()
+ check_toc_regions()
+ check_reference_targets()
+ install_doc.check_prerequisite_regions()
@@ -1378,6 +1442,7 @@ def verify():
f"{len(IGNORE_CANARIES)} ignore canaries clear, "
f"no issue references in {len(shipped_prose())} shipped document(s) or command help, "
f"every docs/ file in one of its {len(DOCS_AREAS)} areas, "
f"the command reference under {reference_doc.REFERENCE_DIR}/ current, "
f"tables of contents current and every link resolving on "
f"{len(toc.target_files())} reference file(s), "
f"{install_doc.INSTALL_DOC} in step with tools/prerequisites.txt and "
@@ -1569,6 +1634,8 @@ def prerequisites_command(
),
see_also=(
"`tools/README.md` § Adding a command - how a command gets its record",
"`wikitool docs reference` - the same records, rendered for a person under "
"`docs/reference/`",
"`wikitool docs verify` - checks the region is current",
),
))
@@ -1597,3 +1664,91 @@ def contract_command(
CLI_README.write_text(after, encoding="utf-8", newline="\n")
success(f"Regenerated the command region in {rel_path(CLI_README)}.")
@app.command("reference")
@cli_contract.record(cli_contract.CommandRecord(
path="docs reference",
summary="Regenerate the human command reference under `docs/reference/`.",
synopsis=(cli_contract.Variant(usage="docs reference [--apply]"),),
properties=cli_contract.Properties(
effect=cli_contract.Effect.WRITE,
idempotent=cli_contract.Idempotent.YES,
atomic="`--apply` rewrites each page in place, one at a time and idempotently, so a "
"re-run after an interruption converges; the dry-run form is read-only",
budget=cli_contract.Budget.COUNTED,
),
notes=(
"Rewrites the generated region of every page of the human command reference from the "
"`cli_contract` records: one `docs/reference/commands/<group>.md` per `cli_contract.GROUPS` "
"entry (`<!-- wikitool:reference-commands -->`), the overview in "
"`docs/reference/commands/README.md` (`<!-- wikitool:reference-commands-index -->`), and "
"the gates and exit-42 causes in `docs/reference/exit-codes.md` "
"(`<!-- wikitool:reference-exit-codes -->`).",
"Renders each command's summary, synopsis, options, examples, notes, properties in words, "
"exit codes with their causes and see-also - but not the ON FAILURE reactions or NEVER, "
"which address an agent; `tools/CONTRACT.md` keeps those.",
"Refreshes the table of contents of each page it writes in the same step.",
"Never creates a page or places a region: a page's title and one-sentence head are "
"written by hand. A group's page is named after the group - lower case, every run of "
"other characters one hyphen.",
"Dry-run by default (lists the pages that would change); `--apply` writes.",
"`docs verify` checks the result stays current.",
),
failures=(cli_contract.Failure(
cause="A reference page is missing, or lacks its region's marker pair",
reaction="Not transient - create the page the message names with a title, a one-sentence "
"head and the marker pair, or add the marker pair below its head, then retry",
),),
examples=(
"tools/wikitool docs reference",
"tools/wikitool docs reference --apply",
),
never=(
"Never hand-edit a reference region - change the record in code and re-run this.",
),
see_also=(
"`wikitool docs contract` - the same records, rendered for an agent into `tools/CONTRACT.md`",
"`wikitool docs verify` - checks every reference region is current",
"`tools/README.md` § Adding a command - how a command gets its record",
),
))
def reference_command(
apply: bool = typer.Option(False, "--apply", help="Write changes; default is dry-run (preview only)"),
):
"""Regenerate the human command reference under `docs/reference/`."""
missing = []
changed = []
for region in reference_doc.regions():
path = config.ROOT / region.path
if not path.is_file():
missing.append(f"{region.path} (create it with `{blocks.open_marker(region.name)}`)")
continue
before = path.read_text(encoding="utf-8")
after = reference_doc.refresh(before, region)
if after is None:
missing.append(f"{region.path} (add `{blocks.open_marker(region.name)}`)")
elif after != before:
changed.append((path, after))
if missing:
fail(
"These reference pages cannot be filled:\n"
+ "\n".join(f"- {entry}" for entry in missing)
+ "\nGive each a title, a one-sentence head and the marker pair "
"(opening and closing), then re-run."
)
if not changed:
success("Every reference page is already current.")
return
for path, after in changed:
typer.echo(rel_path(path))
if apply:
path.write_text(after, encoding="utf-8", newline="\n")
if apply:
success(f"Regenerated {len(changed)} reference page(s).")
else:
typer.echo(f"\n{len(changed)} page(s) would change. Re-run with --apply to write.")
tools/chemenu/reference_doc.py
+253
View File
@@ -0,0 +1,253 @@
"""The human command reference under `docs/reference/`, generated from the records.
`cli_contract` is the one source of what a command does; `tools/CONTRACT.md`
renders it for an agent, and this module renders it for a person. The two
differ on purpose: a record's ON FAILURE reactions and its NEVER lines tell an
agent what to do ("show the output verbatim and stop"), not a person what
happens, so they stay out here - only each failure's cause is kept, under the
exit code it produces. Properties are spelled out in words rather than as the
index's `budget:counted` tokens.
Every file here is half human, half generated, like `INSTALL.md`'s
prerequisites lists: a title and a one-sentence head someone wrote, then one
`<!-- wikitool:reference-... -->` region `docs reference --apply` rewrites. The
tool never creates a page or places a region - a missing one is reported - so
a group added to `cli_contract.GROUPS` gets its page, and its head, from a
person.
`regions()` is the one list both `docs reference` and `docs verify` walk; a
further generated reference page is one more entry in it.
"""
from __future__ import annotations
import re
from pathlib import Path
from typing import Callable, NamedTuple, Optional
from chemenu import blocks, cli_contract, config, toc
REFERENCE_DIR = "docs/reference"
COMMANDS_DIR = f"{REFERENCE_DIR}/commands"
COMMANDS_INDEX = f"{COMMANDS_DIR}/README.md"
EXIT_CODES = f"{REFERENCE_DIR}/exit-codes.md"
COMMANDS_REGION = "reference-commands"
INDEX_REGION = "reference-commands-index"
EXIT_CODES_REGION = "reference-exit-codes"
_SLUG_RE = re.compile(r"[^a-z0-9]+")
class Region(NamedTuple):
"""One generated region: the file it lives in (root-relative), the region's
name, and what renders its content."""
path: str
name: str
render: Callable[[], str]
def group_slug(title: str) -> str:
"""The page name of a `GROUPS` entry: lower case, every run of anything
else than a letter or digit one hyphen (`Types, instructions and docs` ->
`types-instructions-and-docs`)."""
return _SLUG_RE.sub("-", title.lower()).strip("-")
def group_page(title: str) -> str:
return f"{COMMANDS_DIR}/{group_slug(title)}.md"
def command_anchor(path: str) -> str:
"""The anchor of a command's `## `<path>`` heading on its group page."""
return toc.anchor(f"`{path}`")
# ---------------------------------------------------------------------------
# Rendering
# ---------------------------------------------------------------------------
_BUDGET_WORDS = {
cli_contract.Budget.COUNTED: "yes",
cli_contract.Budget.EXEMPT: "no",
cli_contract.Budget.EXEMPT_WITHOUT_ARGS: "only when called with arguments",
}
def _yes_no(value: bool) -> str:
return "yes" if value else "no"
def properties_lines(props: cli_contract.Properties) -> list[str]:
"""A record's properties in words, one list item each."""
lines = [
f"- Changes files: {_yes_no(props.effect == cli_contract.Effect.WRITE)}",
f"- Idempotent: {_yes_no(props.idempotent == cli_contract.Idempotent.YES)}",
f"- Atomic: {props.atomic}",
f"- Counts toward the session call budget: {_BUDGET_WORDS[props.budget]}",
f"- Can reach the network: {_yes_no(props.network == cli_contract.Network.YES)}",
]
if props.gates:
lines.append(
f"- Gates: {', '.join(props.gates)} - see [exit codes and gates](../exit-codes.md)"
)
return lines
def render_command(rec: cli_contract.CommandRecord, options_text: str = "") -> str:
"""One command's section on its group page."""
lines = [f"## `{rec.path}`", "", rec.summary, ""]
lines += ["**Synopsis**", ""]
for variant in rec.synopsis:
note = f" - {variant.notes}" if variant.notes else ""
lines.append(f"- `wikitool {variant.usage}`{note}")
lines.append("")
if options_text.strip():
lines += ["**Options**", "", "```text", options_text.rstrip(), "```", ""]
if rec.examples:
lines += ["**Examples**", "", "```bash", *rec.examples, "```", ""]
lines += ["**Behaviour**", ""]
lines += [f"- {note}" for note in rec.notes]
lines.append("")
lines += ["**Properties**", "", *properties_lines(rec.properties), ""]
lines += ["**Exit codes**", ""]
lines += [f"- `{code}` - {cause}" for code, cause in cli_contract.exit_status_entries(rec)]
lines.append("")
if rec.see_also:
lines += ["**See also**", ""]
lines += [f"- {item}" for item in rec.see_also]
lines.append("")
return "\n".join(lines).rstrip() + "\n"
def render_group(
paths: tuple[str, ...],
records: dict[str, cli_contract.CommandRecord],
options: dict[str, str],
) -> str:
"""The region body of one group page: its commands, in `GROUPS` order."""
return "\n".join(
render_command(records[path], options.get(path, "")) for path in paths if path in records
).rstrip() + "\n"
def render_index(
records: dict[str, cli_contract.CommandRecord],
groups: cli_contract.GroupsType,
) -> str:
"""The region body of the commands README: every group, every command,
its one-sentence summary, linked to its section."""
lines: list[str] = []
for title, paths in groups:
page = f"{group_slug(title)}.md"
lines += [f"## {title}", "", f"Full records: [{page}]({page})", ""]
for path in paths:
if path in records:
lines.append(
f"- [`{path}`]({page}#{command_anchor(path)}) - {records[path].summary}"
)
lines.append("")
return "\n".join(lines).rstrip() + "\n"
def _command_link(path: str, groups: cli_contract.GroupsType) -> str:
title = cli_contract.group_of(path, groups)
if title is None:
return f"`{path}`"
return f"[`{path}`](commands/{group_slug(title)}.md#{command_anchor(path)})"
def render_exit_codes(
records: dict[str, cli_contract.CommandRecord],
groups: cli_contract.GroupsType,
) -> str:
"""The region body of `exit-codes.md`: which command declares which gate,
and every cause of an exit 42. What an exit code means is linked, not
repeated - `AGENTS.md` is where that rule lives."""
ordered = [path for path in cli_contract.grouped_paths(groups) if path in records]
lines = [
"What each exit code means, and how a caller reacts to it, is "
"[AGENTS.md § Tool error contract](../../AGENTS.md#tool-error-contract). What a gate is "
"and why it is enforced in code is [AGENTS.md § Gates](../../AGENTS.md#gates). This page "
"lists which commands declare a gate, and every cause for which a command exits 42.",
"",
"## Gates by command",
"",
]
gates: dict[str, list[str]] = {}
for path in ordered:
for gate in records[path].properties.gates:
gates.setdefault(gate, []).append(path)
for gate in sorted(gates):
lines += [f"### {gate}", ""]
lines += [f"- {_command_link(path, groups)}" for path in gates[gate]]
lines.append("")
lines += ["## Exit 42 causes", ""]
for path in ordered:
causes = [cause for code, cause in cli_contract.exit_status_entries(records[path]) if code == 42]
lines += [f"- {_command_link(path, groups)}: {cause}" for cause in causes]
return "\n".join(lines).rstrip() + "\n"
# ---------------------------------------------------------------------------
# The generated regions, and writing one
# ---------------------------------------------------------------------------
def regions(
records: Optional[dict[str, cli_contract.CommandRecord]] = None,
groups: Optional[cli_contract.GroupsType] = None,
options: Optional[dict[str, str]] = None,
) -> list[Region]:
"""Every generated region under `docs/reference/`, in a stable order.
`options` defaults to the real CLI's rendered OPTIONS blocks; a test
passing its own records passes its own options too (or `{}`)."""
records = records if records is not None else cli_contract.all_records()
groups = groups if groups is not None else cli_contract.GROUPS
if options is None:
from chemenu import cli # deferred: `cli` imports every command module, this one's caller included
options = cli.options_texts([p for p in cli_contract.grouped_paths(groups) if p in records])
found = [
Region(group_page(title), COMMANDS_REGION,
lambda paths=paths: render_group(paths, records, options))
for title, paths in groups
]
found.append(Region(COMMANDS_INDEX, INDEX_REGION, lambda: render_index(records, groups)))
found.append(Region(EXIT_CODES, EXIT_CODES_REGION, lambda: render_exit_codes(records, groups)))
return found
def refresh(text: str, region: Region) -> Optional[str]:
"""`text` with `region` regenerated and its table of contents brought in
line, or None when the file carries no such region to fill.
The TOC is refreshed in the same step because the generated `##` headings
are what it lists - without it, every regeneration that added a command
would leave `docs verify` reporting a stale TOC until `docs toc --apply`
ran as well."""
if blocks.find(text, region.name) is None:
return None
content = region.render().strip("\n")
wrapped = blocks.open_marker(region.name) + "\n" + content + "\n" + blocks.close_marker(region.name)
return toc.upsert(blocks.replace(text, region.name, wrapped))
def orphan_pages(groups: Optional[cli_contract.GroupsType] = None) -> list[Path]:
"""Pages under `docs/reference/commands/` no group in `groups` names."""
groups = groups if groups is not None else cli_contract.GROUPS
directory = config.ROOT / COMMANDS_DIR
if not directory.is_dir():
return []
expected = {Path(group_page(title)).name for title, _ in groups} | {Path(COMMANDS_INDEX).name}
return sorted(p for p in directory.glob("*.md") if p.name not in expected)
tools/chemenu/tests/test_reference_doc.py
+288
View File
@@ -0,0 +1,288 @@
import re
import pytest
from typer.testing import CliRunner
from chemenu import blocks, cli, cli_contract, config, reference_doc, toc
from chemenu.cli import app
from chemenu.commands import dist_cmd, docs_verify
runner = CliRunner()
# --- the real tree --------------------------------------------------------
def _group_pages() -> dict[str, str]:
return {
title: (config.ROOT / reference_doc.group_page(title)).read_text(encoding="utf-8")
for title, _ in cli_contract.GROUPS
}
def test_the_real_reference_is_current():
"""What a session that changed a record and forgot `docs reference --apply`
is caught by."""
assert docs_verify.check_reference_regions() == []
def test_every_group_has_one_page_and_every_command_one_section():
pages = _group_pages()
for path in cli_contract.all_records():
heading = f"## `{path}`\n"
hits = [title for title, text in pages.items() if text.count(heading) == 1]
assert len(hits) == 1, path
assert sum(text.count(heading) for text in pages.values()) == 1, path
def test_group_slugs_are_unique():
slugs = [reference_doc.group_slug(title) for title, _ in cli_contract.GROUPS]
assert len(slugs) == len(set(slugs))
def test_no_page_carries_an_agent_addressed_field():
"""ON FAILURE reactions and NEVER tell an agent what to do; the human
reference keeps only the cause of each failure."""
text = "\n".join(_group_pages().values())
text += (config.ROOT / reference_doc.EXIT_CODES).read_text(encoding="utf-8")
for rec in cli_contract.all_records().values():
for failure in rec.failures:
if failure.reaction:
assert failure.reaction not in text, (rec.path, failure.reaction)
for line in rec.never:
assert line not in text, (rec.path, line)
def test_options_match_the_options_section_of_h():
texts = cli.options_texts(sorted(cli_contract.all_records()))
for path, options in texts.items():
result = runner.invoke(app, [*path.split(" "), "-h"])
assert result.exit_code == 0, (path, result.output)
if options.strip():
assert f"OPTIONS\n{options.rstrip()}\n\n" in result.output, path
else:
assert "\nOPTIONS\n" not in result.output, path
def _section(markdown: str, heading: str) -> str:
start = markdown.index(f"## {heading}\n")
end = markdown.find("\n## ", start + 1)
return markdown[start:end if end != -1 else None]
def test_exit_codes_names_every_gate_and_links_rather_than_repeats_agents_md():
page = (config.ROOT / reference_doc.EXIT_CODES).read_text(encoding="utf-8")
for rec in cli_contract.all_records().values():
for gate in rec.properties.gates:
assert f"### {gate}\n" in page, gate
assert "(../../AGENTS.md#tool-error-contract)" in page
assert "(../../AGENTS.md#gates)" in page
agents = (config.ROOT / "AGENTS.md").read_text(encoding="utf-8")
for heading in ("Tool error contract", "Gates"):
for sentence in re.split(r"(?<=[.:])\s+", _section(agents, heading)):
sentence = sentence.strip()
if len(sentence) >= 30:
assert sentence not in page, sentence
def test_the_reference_pages_survive_dist_export_unchanged():
"""`dist export` strips `dist:strip` spans from every shipped text file; a
record quoting the marker pair verbatim would be cut in the exported page
and the fresh instance's `docs verify` would call it stale."""
plan = dist_cmd.build_plan()
for region in reference_doc.regions():
assert plan[region.path].content == (config.ROOT / region.path).read_text(encoding="utf-8")
# --- a fixture tree -------------------------------------------------------
GROUPS = (("Fixture things", ("frobnicate",)), ("Other", ("gate keep",)))
def _record(path: str, gates: tuple[str, ...] = (), summary: str = "Does a thing.") -> cli_contract.CommandRecord:
failures = [cli_contract.Failure(cause="It broke", reaction="Show the user and stop")]
if gates:
failures.append(cli_contract.Failure(cause="Needs a person", reaction="Wait", code=42))
return cli_contract.CommandRecord(
path=path,
summary=summary,
synopsis=(cli_contract.Variant(usage=f"{path} [--flag]"),),
properties=cli_contract.Properties(
effect=cli_contract.Effect.WRITE,
idempotent=cli_contract.Idempotent.NO,
atomic="Yes",
budget=cli_contract.Budget.EXEMPT_WITHOUT_ARGS,
gates=gates,
),
notes=("Does a thing, mechanically.",),
failures=tuple(failures),
examples=("tools/wikitool frobnicate --flag",),
never=("Never frobnicate twice.",),
see_also=("`wikitool other` - a neighbour",),
)
HEADS = {
"docs/reference/commands/fixture-things.md": "# Fixture things\n\nThings, by hand.\n",
"docs/reference/commands/other.md": "# Other\n\nThe other one.\n",
"docs/reference/commands/README.md": "# Command reference\n\nAll of it.\n",
"docs/reference/exit-codes.md": "# Exit codes and gates\n\nWhere it stops.\n",
}
REGION_OF = {
"docs/reference/commands/fixture-things.md": reference_doc.COMMANDS_REGION,
"docs/reference/commands/other.md": reference_doc.COMMANDS_REGION,
"docs/reference/commands/README.md": reference_doc.INDEX_REGION,
"docs/reference/exit-codes.md": reference_doc.EXIT_CODES_REGION,
}
@pytest.fixture
def tree(tmp_path, monkeypatch):
monkeypatch.setattr(config, "ROOT", tmp_path)
records = {"frobnicate": _record("frobnicate"), "gate keep": _record("gate keep", ("keep",))}
monkeypatch.setattr(cli_contract, "all_records", lambda: dict(records))
monkeypatch.setattr(cli_contract, "GROUPS", GROUPS)
monkeypatch.setattr(cli, "options_texts", lambda paths: {p: " --flag A flag." for p in paths})
for relative, head in HEADS.items():
path = tmp_path / relative
path.parent.mkdir(parents=True, exist_ok=True)
name = REGION_OF[relative]
path.write_text(
f"{head}\n{blocks.open_marker(name)}\n{blocks.close_marker(name)}\n", encoding="utf-8"
)
return tmp_path, records
def _snapshot(root) -> dict[str, str]:
return {
p.relative_to(root).as_posix(): p.read_text(encoding="utf-8")
for p in sorted(root.rglob("*.md"))
}
def test_a_fresh_tree_is_reported_stale_until_applied(tree):
root, _ = tree
issues = docs_verify.check_reference_regions()
assert len(issues) == len(HEADS)
assert all("stale" in issue and "docs reference --apply" in issue for issue in issues)
dry = runner.invoke(app, ["docs", "reference"])
assert dry.exit_code == 0, dry.output
assert "would change" in dry.output
assert docs_verify.check_reference_regions() != []
result = runner.invoke(app, ["docs", "reference", "--apply"])
assert result.exit_code == 0, result.output
assert docs_verify.check_reference_regions() == []
def test_apply_twice_changes_nothing_the_second_time(tree):
root, _ = tree
runner.invoke(app, ["docs", "reference", "--apply"])
first = _snapshot(root)
second_run = runner.invoke(app, ["docs", "reference", "--apply"])
assert second_run.exit_code == 0
assert "already current" in second_run.output
assert _snapshot(root) == first
def test_the_head_survives_apply_byte_for_byte(tree):
root, _ = tree
runner.invoke(app, ["docs", "reference", "--apply"])
for relative, head in HEADS.items():
text = (root / relative).read_text(encoding="utf-8")
assert text.startswith(head), relative
def test_a_changed_record_makes_verify_name_the_page_and_the_command(tree, monkeypatch):
root, records = tree
runner.invoke(app, ["docs", "reference", "--apply"])
records["frobnicate"] = _record("frobnicate", summary="Does a different thing.")
monkeypatch.setattr(cli_contract, "all_records", lambda: dict(records))
issues = docs_verify.check_reference_regions()
assert any(
"docs/reference/commands/fixture-things.md" in issue and "docs reference --apply" in issue
for issue in issues
)
def test_a_generated_page_leaves_out_reactions_and_never(tree):
root, _ = tree
runner.invoke(app, ["docs", "reference", "--apply"])
text = (root / "docs/reference/commands/fixture-things.md").read_text(encoding="utf-8")
assert "It broke" in text
assert "Show the user and stop" not in text
assert "Never frobnicate twice." not in text
assert "- Counts toward the session call budget: only when called with arguments" in text
assert "```text\n --flag A flag.\n```" in text
def test_a_long_page_gets_its_toc_outside_the_generated_region(tree, monkeypatch):
"""The first `##` of a group page is generated; a TOC inserted before it
would sit inside the region and be wiped by the next regeneration."""
root, records = tree
many = tuple(f"cmd{i}" for i in range(12))
for path in many:
records[path] = _record(path)
monkeypatch.setattr(cli_contract, "all_records", lambda: dict(records))
monkeypatch.setattr(cli_contract, "GROUPS", (("Fixture things", many), GROUPS[1]))
assert runner.invoke(app, ["docs", "reference", "--apply"]).exit_code == 0
page = root / "docs/reference/commands/fixture-things.md"
text = page.read_text(encoding="utf-8")
assert toc.needs_toc(text)
assert text.index(blocks.close_marker(toc.REGION_NAME)) < text.index(
blocks.open_marker(reference_doc.COMMANDS_REGION)
)
assert toc.upsert(text) == text
assert docs_verify.check_reference_regions() == []
def test_a_group_without_a_page_fails_and_nothing_is_created(tree, monkeypatch):
root, _ = tree
monkeypatch.setattr(cli_contract, "GROUPS", GROUPS + (("New group", ()),))
issues = docs_verify.check_reference_regions()
assert any("docs/reference/commands/new-group.md is missing" in issue for issue in issues)
result = runner.invoke(app, ["docs", "reference", "--apply"])
assert result.exit_code == 1
assert "docs/reference/commands/new-group.md" in result.output
assert not (root / "docs/reference/commands/new-group.md").exists()
def test_a_page_without_its_marker_pair_fails_and_is_left_alone(tree):
root, _ = tree
page = root / "docs/reference/exit-codes.md"
page.write_text("# Exit codes and gates\n\nNo region here.\n", encoding="utf-8")
issues = docs_verify.check_reference_regions()
assert any("exit-codes.md has no" in issue for issue in issues)
result = runner.invoke(app, ["docs", "reference", "--apply"])
assert result.exit_code == 1
assert page.read_text(encoding="utf-8") == "# Exit codes and gates\n\nNo region here.\n"
def test_a_page_without_a_group_fails(tree):
root, _ = tree
(root / "docs/reference/commands/gone.md").write_text("# Gone\n", encoding="utf-8")
issues = docs_verify.check_reference_regions()
assert any("gone.md belongs to no group" in issue for issue in issues)
def test_a_tree_without_docs_passes(tmp_path, monkeypatch):
monkeypatch.setattr(config, "ROOT", tmp_path)
assert docs_verify.check_reference_regions() == []
def test_exit_codes_lists_each_gate_and_its_exit_42_causes(tree):
root, _ = tree
runner.invoke(app, ["docs", "reference", "--apply"])
page = (root / "docs/reference/exit-codes.md").read_text(encoding="utf-8")
assert "### keep\n\n- [`gate keep`](commands/other.md#gate-keep)" in page
assert "- [`gate keep`](commands/other.md#gate-keep): Needs a person" in page
assert "frobnicate" not in page
def test_the_index_links_every_command_to_its_section(tree):
root, _ = tree
runner.invoke(app, ["docs", "reference", "--apply"])
index = (root / "docs/reference/commands/README.md").read_text(encoding="utf-8")
assert "- [`frobnicate`](fixture-things.md#frobnicate) - Does a thing." in index
assert "- [`gate keep`](other.md#gate-keep) - Does a thing." in index
tools/chemenu/toc.py
+22 -1
View File
@@ -276,6 +276,12 @@ def _slugify(text: str, seen: dict[str, int]) -> str:
return base if count == 0 else f"{base}-{count}"
def anchor(text: str) -> str:
"""The anchor a heading with this text gets, the first time it occurs on
a page - for a link written from outside that page."""
return _slugify(text, {})
def render(headings: list[Heading]) -> str:
"""The TOC region ready to place into a body, or `""` for no headings."""
if not headings:
@@ -289,12 +295,27 @@ def render(headings: list[Heading]) -> str:
return blocks.render(REGION_NAME, HEADING_TEXT, lines)
# Any generated region other than the TOC itself, markers included.
_OTHER_REGION_RE = re.compile(
r"<!-- wikitool:([a-z][a-z0-9-]*) -->.*?<!-- /wikitool:\1 -->", re.DOTALL
)
def _first_level2_offset(body: str) -> int | None:
"""Character offset of the first real (non-fenced) `## ` heading, or None."""
"""Character offset of the first real (non-fenced) `## ` heading, or None.
A heading inside another generated region moves the offset to that
region's opening marker: a page under `docs/reference/` can open with a
generated region whose first line is a `##` heading, and a TOC inserted
there would sit inside a region the next regeneration replaces wholesale.
"""
masked = markdown_code.strip_code_spans(body)
pos = 0
for masked_line in masked.split("\n"):
if masked_line.startswith("## "):
for region in _OTHER_REGION_RE.finditer(masked):
if region.start() < pos < region.end():
return region.start()
return pos
pos += len(masked_line) + 1
return None