torben/chemenu
1
1 Fork 0
Code Issues 40 Pull Requests Actions Packages Releases 41 Wiki Activity

Mirror docs at v10.1.0

Wiki Mirror committed 2026-10-16 14:43:37 +00:00
1 parent 7b705fa5c4
commit d7a9159155
56 files changed
+384 -180
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Dev-Development.md
+48 -14
@@ -1,4 +1,4 @@
*Generated from [`docs/dev/development.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/dev/development.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/dev/development.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/dev/development.md) at v10.1.0 – edit the repository, not this wiki.*
# Developing this stack
@@ -40,17 +40,29 @@ cd chemenu
Then have the agent run `instructions/bootstrap.md`: the preflight, then
`tools/wikitool instructions sync`. The agent's side of it - what differs here from an instance,
and how `dist export` runs as a test tool - is
[instructions/dev/dev-setup.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/dev/dev-setup.md).
[instructions/dev/dev-setup.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/dev/dev-setup.md).
Real documents for `raw derive` live in [testfiles/](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/testfiles/README.md) and are
committed, so CI derives them too. The repository is public: that README says what may go in.
Real documents for `raw derive` do not live in this repository. The corpus and its test harness
are the private repository `torben/chemenu-testfiles`, cloned by hand into `testfiles/` (gitignored
here, deliberately not a submodule, so a clone without access works unchanged):
```bash
git clone ssh://git@gitea.nehmer.net:10101/torben/chemenu-testfiles.git testfiles
cd tools && .venv/bin/python -m pytest -c pytest.ini ../testfiles/harness
```
It is private because the corpus is large (some 76 MB; one book takes 48 s per derivation) and
holds material that may not sit in a public repository, such as publishers' books and vendors'
documents. Without the clone, `pytest` runs unchanged and the real-document tests are simply not
there; the README of the private repository says what is in it, what each file's origin is and
whether it could be public. The tests run in their own workflow, see [The CI half](#the-ci-half).
What a clone has and an instance does not:
- **The demo corpus.** Some 170 pages documenting the stack itself: gates, lint, versioning,
search, the wiki pattern. It is a testbed and a walkable example, not a productive knowledge
base. What may be changed in it is governed by
[instructions/dev/corpus-policy.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/dev/corpus-policy.md).
[instructions/dev/corpus-policy.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/dev/corpus-policy.md).
- **A demo persona** in `USER.md`/`SOUL.md`. `doctor` reports both as filled in. They describe
the demo setup, not you.
- **Telemetry on.** Without `.wikitool-release.json` the clone is the measuring station the stack
@@ -91,7 +103,7 @@ only to downloads on the same host as the feed.
Between two releases the stack carries **one** running version candidate instead of a new
number per bump. The full model - where the state lives, how escalation works, why a number is
only consumed by a release - is in
[instructions/dev/version-parts.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/dev/version-parts.md) and
[instructions/dev/version-parts.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/dev/version-parts.md) and
[docs/explanation/version-model.md](Explanation-Version-model). Here only the sequence, in
the order a session actually goes through it:
@@ -112,7 +124,7 @@ the order a session actually goes through it:
lines where they apply). A smaller change needs nothing beyond its bump title; a larger one
gets a paragraph for its topic, and a short summary is added before the release. The form, and
the size budget `docs verify` and `version release` check, are in
[instructions/dev/version-parts.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/dev/version-parts.md) § The candidate
[instructions/dev/version-parts.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/dev/version-parts.md) § The candidate
model.
3. **Run the checks before anything is published:**
@@ -134,7 +146,7 @@ the order a session actually goes through it:
bump titles gets a summarising heading. Refused when the candidate collected two or more bumps
and the summary from step 2 is still missing - a candidate with exactly one bump is exempt -
and likewise when the entry is over its size budget. Commits and pushes nothing (invariant 5
in [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md)).
in [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md)).
5. **Publish moves `VERSION` on `main`.**
@@ -143,7 +155,7 @@ the order a session actually goes through it:
```
The Mass-Update Gate and the Publish-Remote Gate apply as on every other publish - see
[instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/gates.md).
[instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/gates.md).
6. **CI does the rest.** `.gitea/workflows/release.yml` reacts to every push that moves
`VERSION`: a `VERSION` with a suffix (a candidate) makes the job skip cleanly before it even
@@ -166,12 +178,12 @@ the order a session actually goes through it:
own, so a failing mirror leaves the release and its tag standing and marks only itself red.
The three check commands are in step 3 above; what each of them checks is in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/CONTRACT.md), where `docs verify` holds it against the actual
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/CONTRACT.md), where `docs verify` holds it against the actual
CLI. It is deliberately **not** repeated here: a second description of the same commands is
exactly the copy that drifts (AGENTS.md invariant 8), and this document lies outside the files
the command-record check of `docs verify` covers - so a drift here would go unnoticed. What
`pytest` expects of the developer at this point is in
[instructions/dev/testing-conventions.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/dev/testing-conventions.md).
[instructions/dev/testing-conventions.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/dev/testing-conventions.md).
## The CI half
@@ -183,6 +195,28 @@ the download. `.gitea/workflows/nightly.yml` is the drift check against time rat
a commit. `.gitea/workflows/release.yml` is step 6 above, and `.gitea/workflows/wiki-mirror.yml`
refills the Gitea wiki by hand (see below).
`.gitea/workflows/converters.yml` runs the converters against the real documents of the private
repository (the clone above, run with the checkout's `tools/.venv`). It starts by itself on a push
to `main` that touches the converter surface - `tools/chemenu/filters/`, `raw/derive.py`,
`raw/accept.py`, `tests/test_raw_derive*.py`, `tests/binary_formats.py`, `tools/requirements.txt`
and the workflow itself - and by hand (`workflow_dispatch`), which is how to run it after a change
in the private repository alone: a push there starts nothing here. It always tests `main` of the
private repository, so a converter change that alters an expected output is two commits, the
private one first. Before a release without suffix, `release.yml` runs the same steps as a job its
release job waits for; a red harness ends the run before a release or a tag exists, and a
`-beta.N` candidate skips it. The job reads the private repository with the per-job token, which
works only because the private repository's own Actions settings grant `torben/chemenu` access;
without that grant the checkout step fails with "Determining the default branch … not found",
which reads like a missing repository rather than a missing permission.
That workflow shows that the corpus still derives, not that the derivation is good. The second
question is a session's: `harness/compare/` in the same private repository derives every corpus
file with the checkout's filters and with reference tools (pymupdf4llm, markitdown, pandoc, each in
a throwaway environment of its own, never in `tools/.venv`), measures wording fidelity, markup
overhead, missing semantics, structure, CHECK precision and runtime, and reports the difference to
a committed baseline. When a session runs it and what it does with the result is
[instructions/dev/converter-comparison.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/dev/converter-comparison.md).
Three more workflows carry the live tests of the tracker adapters (Super Productivity, CalDAV):
`ci.yml` runs the CalDAV half in a step of its own against Radicale as a process,
`tracker-live.yml` runs nightly and covers both providers, and `sp-live-image.yml` builds the
@@ -191,7 +225,7 @@ Super Productivity version. The image follows the desktop clients' update channe
version. Once, after the very first push, the package has to be linked to the `torben/chemenu`
repository by hand. What the suite writes, how to run it against a tracker of your own, and what
a red run means is in
[instructions/dev/tracker-testing.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/dev/tracker-testing.md).
[instructions/dev/tracker-testing.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/dev/tracker-testing.md).
### The Gitea wiki
@@ -289,5 +323,5 @@ automatically; `stack-build` and `stack-close` are started only by the operator,
(`/stack-build #N`, `/stack-close`) - exactly where the choice falls whether to continue in the
same session or a new one, and on which model. None of the three offers a model switch in the
middle of a session. Rules and the phase table are in
[instructions/dev/stack-mode.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/dev/stack-mode.md), the issue tracker itself
in [instructions/dev/issue-tracking.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/dev/issue-tracking.md).
[instructions/dev/stack-mode.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/dev/stack-mode.md), the issue tracker itself
in [instructions/dev/issue-tracking.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/dev/issue-tracking.md).
Explanation-Architecture.md
+4 -4
@@ -1,4 +1,4 @@
*Generated from [`docs/explanation/architecture.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/architecture.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/architecture.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/architecture.md) at v10.1.0 – edit the repository, not this wiki.*
# Architecture
@@ -43,7 +43,7 @@ citation ids, schema enum values, tags, commands, paths and code. So `GitOps Own
German section heading sit in the same page without contradiction.
Which lines count as identifiers is the stack's rule,
[kb/CONTRACT.md § Language and identifiers](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/kb/CONTRACT.md#language-and-identifiers).
[kb/CONTRACT.md § Language and identifiers](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/kb/CONTRACT.md#language-and-identifiers).
*Which language* the prose is in, and what the tool-owned headings are called, is the instance's own
`kb/CONVENTIONS.md`. That is a per-instance decision, not a property of the pattern, so it lives in a
file the instance owns rather than in one the stack ships. A new instance picks its language when it
@@ -229,12 +229,12 @@ same record is rendered four ways:
command;
- a command that fails with exit 1 prints the record's reactions on stderr, right under its `ERROR`
line;
- [tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/CONTRACT.md) holds a generated copy for agents, together with the
- [tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/CONTRACT.md) holds a generated copy for agents, together with the
maintenance schedule;
- [the command reference](Reference-Commands) is a generated copy for humans.
`tools/wikitool docs verify` checks the copies, and every command's flags, against the CLI in both
directions. [tools/README.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/README.md) is the other half: how the CLI is built and how
directions. [tools/README.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/README.md) is the other half: how the CLI is built and how
to add a command. `AGENTS.md` holds the invariants that say when each command is mandatory.
The wiki is also a git repository, and `tools/wikitool publish` is the one way its changes are
Explanation-Configuration-tiers.md
+3 -3
@@ -1,4 +1,4 @@
*Generated from [`docs/explanation/configuration-tiers.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/configuration-tiers.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/configuration-tiers.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/configuration-tiers.md) at v10.1.0 – edit the repository, not this wiki.*
# Configuration tiers
@@ -6,7 +6,7 @@ Settings are split into three tiers: what belongs to the instance (`wikitool.jso
what belongs to one checkout (`.wikitool-local.json`, gitignored), and secrets, which sit
outside the working tree and are reached only by reference. The files, their fields and the
command that writes them are in [the configuration reference](Reference-Configuration#tiered-settings);
the rules an agent follows are in [AGENTS.md § Settings](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#settings). This page is
the rules an agent follows are in [AGENTS.md § Settings](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#settings). This page is
about why it is built this way, and what it does not do.
<!-- wikitool:toc -->
@@ -110,6 +110,6 @@ None of these covers a website session from `raw login`, which lives in the secr
Whoever can read that file can use the website as the user, until the session expires or the user
signs out there.
The rule in [AGENTS.md § Settings](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#settings) - a secret never passes through the
The rule in [AGENTS.md § Settings](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#settings) - a secret never passes through the
agent - is an instruction, not a barrier. The tiers make following it the easy path. They do not
make breaking it impossible.
Explanation-Knowledge-and-commitment.md
+6 -6
@@ -1,4 +1,4 @@
*Generated from [`docs/explanation/knowledge-and-commitment.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/knowledge-and-commitment.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/knowledge-and-commitment.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/knowledge-and-commitment.md) at v10.1.0 – edit the repository, not this wiki.*
# Why knowledge and commitments are two layers
@@ -6,8 +6,8 @@ Chemenu compiles knowledge into `kb/`, and it also tracks what its operator has
Those look like one subject - both are "things about my projects" - and the stack deliberately
keeps them apart: `kb/gtd/` holds one page per initiative, an external task tracker holds the
open items, and the only thing that crosses between them is a name. This page is about why that
line was drawn there. The rules that follow from it live in [kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/kb/CONTRACT.md)
and the `review`, `new` and `task new` records in [tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/CONTRACT.md).
line was drawn there. The rules that follow from it live in [kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/kb/CONTRACT.md)
and the `review`, `new` and `task new` records in [tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/CONTRACT.md).
<!-- wikitool:toc -->
## Contents
@@ -105,9 +105,9 @@ tracker item, never a page; a source that also carries knowledge gets that knowl
through the ordinary page-creation commands, as a separate step. The two are never one
transaction the way `new project`'s tracker-then-page order is within a single command - they are
two independent writes a skill sequences, tracker first, so a failure creating the item leaves no
page and no promoted source material behind it. That holds for an ordinary source; a source large
or broad enough to run through the large-tree procedure instead promotes ahead of its own
per-unit commitment decision, because that procedure hands its units through a workshop directory
page and no promoted source material behind it. That holds for an ordinary source; a source large,
broad or long enough to run through the large-tree procedure instead promotes ahead of its own
commitment decision, because that procedure hands its units through a workshop directory
that needs them already promoted to address them at all - the same raw-file-without-page state
the ordinary case avoids becomes, there, the expected condition for as long as the run takes. And
a failure on the knowledge side afterwards is exactly the ordinary "a source without a page" state
Explanation-Language-boundaries.md
+6 -6
@@ -1,16 +1,16 @@
*Generated from [`docs/explanation/language-boundaries.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/language-boundaries.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/language-boundaries.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/language-boundaries.md) at v10.1.0 – edit the repository, not this wiki.*
# Language Boundaries
Two languages run through this repo at once. `kb/` is written in whatever language the instance
chose - German here, and the value lives in `kb/CONVENTIONS.md`'s `language:`. Everything that
tells an agent what to do - [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md), every `CONTRACT.md`, everything under
tells an agent what to do - [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md), every `CONTRACT.md`, everything under
`instructions/` - is written in English, in every instance, whatever the first value says. So
is the stack's own documentation for humans, for a different reason (§ "The stack's human
documentation").
The rules themselves are in [AGENTS.md § File naming](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#file-naming) (the language
of a file) and [AGENTS.md § Conversation language](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#conversation-language) (the
The rules themselves are in [AGENTS.md § File naming](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#file-naming) (the language
of a file) and [AGENTS.md § Conversation language](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#conversation-language) (the
language of what an agent says). This page holds the
part that is not a rule: why the line runs where it does, why the English half is not a setting,
and which argument for it turned out to be wrong.
@@ -43,7 +43,7 @@ guidance is still instruction addressed to an agent, and reads exactly like the
four types the stack ships. Ownership says "yours"; the audience has not moved at all.
So the question a line answers is not *whose file is this* but *who reads this line*, which is
the same cut [kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/kb/CONTRACT.md#language-and-identifiers) already makes inside a
the same cut [kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/kb/CONTRACT.md#language-and-identifiers) already makes inside a
single page between prose and identifiers - applied one level up, to the halves of a document.
Ownership decides who may change a sentence. The reader decides what language it is in. The two
questions were answered together for as long as they happened to agree.
@@ -149,7 +149,7 @@ beside the template, but that split was made for ownership reasons - so an upgra
the guidance without overwriting what the instance chose - and it leaves this page's argument
untouched: each part is still written in the language its own reader needs, and a type-spec that
declares no `guidance:` keeps both halves in one file with exactly the same rule applying inside
it. [types/type-spec.md § Who owns a type-spec](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/types/type-spec.md#who-owns-a-type-spec) has
it. [types/type-spec.md § Who owns a type-spec](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/types/type-spec.md#who-owns-a-type-spec) has
the split as a table, and [ownership-and-templates.md](Explanation-Ownership-and-templates) § "Where the
file boundary used to strain" has what it cost to keep two audiences in one file for as long as
it did.
Explanation-Model-and-effort-selection.md
+2 -2
@@ -1,10 +1,10 @@
*Generated from [`docs/explanation/model-and-effort-selection.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/model-and-effort-selection.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/model-and-effort-selection.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/model-and-effort-selection.md) at v10.1.0 – edit the repository, not this wiki.*
# Choosing a Claude Code model and effort level
Claude Code exposes three choices this repo has an opinion on: which model a session itself
runs as, what model a spawned subagent gets, and which `/code-review` effort level to pick.
None of them are enforced anywhere - the gates in [instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/gates.md)
None of them are enforced anywhere - the gates in [instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/gates.md)
are code precisely because a model cannot be talked out of them
([why-gates-are-code.md](Explanation-Why-gates-are-code) makes that argument for gates; this page applies
the same axis to who is holding the keyboard). What follows is a reference for making that choice
Explanation-Ownership-and-templates.md
+8 -8
@@ -1,11 +1,11 @@
*Generated from [`docs/explanation/ownership-and-templates.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/ownership-and-templates.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/ownership-and-templates.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/ownership-and-templates.md) at v10.1.0 – edit the repository, not this wiki.*
# Ownership and Templates
Chemenu ships two kinds of files side by side, and at a glance they look the same: both are
plain markdown, both sit in the repo root or under `kb/`, both get read at session start. But a
stack upgrade treats them completely differently. Some - [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md),
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/kb/CONTRACT.md), the per-stage contracts - are identical in every instance
stack upgrade treats them completely differently. Some - [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md),
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/kb/CONTRACT.md), the per-stage contracts - are identical in every instance
that runs this stack and are the next release's to replace (with one caveat about local edits,
below). Others - `USER.md`,
`SOUL.md`, `kb/CONVENTIONS.md`, `ENVIRONMENT.md` - describe one particular instance, and
@@ -27,7 +27,7 @@ overwriting them would silently erase a choice someone made on purpose.
The stack-owned files describe how the tool works. `kb/CONTRACT.md` opens by saying it holds
what `tools/wikitool` enforces or what follows mechanically from how it operates - see
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/kb/CONTRACT.md), lines 10-13. That kind of statement doesn't vary by
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/kb/CONTRACT.md), lines 10-13. That kind of statement doesn't vary by
instance: the compiler behaves the same way regardless of who is running it, so the sentence
describing that behavior can be copied byte-for-byte into every checkout without becoming
wrong anywhere.
@@ -35,7 +35,7 @@ wrong anywhere.
The instance-owned files describe a choice: which language pages are written in, what tone the
agent takes, who the operator is, which git remote is authoritative, which MCP servers are
reachable. None of that follows from the tool's mechanics - two instances of the identical
stack can answer all of these differently and both be correct. [AGENTS.md § Personalization](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#personalization)
stack can answer all of these differently and both be correct. [AGENTS.md § Personalization](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#personalization)
frames the split the same way for `kb/CONTRACT.md` versus `kb/CONVENTIONS.md`: "the split is by
who may change the sentence, not by what it is about." A rule about page structure could in
principle have been written per-instance too, but then every instance answering "not German" to
@@ -61,7 +61,7 @@ the file itself already answers that.
For a while the boundary was written down as a list of paths - once in `dist_cmd.py`, once in
the merge procedure a private instance was told to run by hand, and once in the check that
procedure ended with. Three copies of one fact, which is the shape [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md)
procedure ended with. Three copies of one fact, which is the shape [AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md)
invariant 8 exists to forbid, and they drifted exactly as predicted: the hand-run procedure was
still naming three paths after the collection contracts had moved to the instance's side of the
line, so it discarded upstream changes to files it had never heard of, while its own final check
@@ -83,7 +83,7 @@ The mechanism for instance-owned content is a `.template` file the distribution
of the real one - `USER.md.template`, `SOUL.md.template`, `kb/CONVENTIONS.md.template`,
`ENVIRONMENT.md.template`. An alternative would have been to ship nothing at all and let a
brand-new instance start from a blank page. The template exists because a blank page doesn't
tell [instructions/setup-instance.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/setup-instance.md) what shape the answer
tell [instructions/setup-instance.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/setup-instance.md) what shape the answer
should take, and it gives nothing for a validator to check afterward.
A template carries a placeholder value - a sentinel - in the fields that need a real answer.
@@ -91,7 +91,7 @@ Setup interviews the operator and replaces the sentinel with what they actually
gives `doctor` a mechanical way to tell "personalized" from "not yet": a file that still
contains the sentinel hasn't been through setup, regardless of whether the file exists. That's
also why `ENVIRONMENT.md` only warrants a WARN rather than a FAIL when absent - see
[AGENTS.md § Environment](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#environment) - while a missing or unfilled
[AGENTS.md § Environment](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#environment) - while a missing or unfilled
`USER.md`/`SOUL.md`/`kb/CONVENTIONS.md` is a harder failure: `ENVIRONMENT.md` describes one
checkout among possibly several and is gitignored for that reason, so its absence is a normal
state rather than a sign setup was skipped.
Explanation-Pipeline-rationale.md
+19 -8
@@ -1,16 +1,16 @@
*Generated from [`docs/explanation/pipeline-rationale.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/pipeline-rationale.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/pipeline-rationale.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/pipeline-rationale.md) at v10.1.0 – edit the repository, not this wiki.*
# Why the pipeline has four stages
Chemenu could, in principle, be one directory: drop a file in, ask a question, get an answer
computed fresh each time. It isn't built that way. The pipeline in
[AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#routing) - `raw/` -> `[types/ + tools/]` -> `kb/` -> `reports/`, with
[AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#routing) - `raw/` -> `[types/ + tools/]` -> `kb/` -> `reports/`, with
`work/` alongside rather than inside it - separates *material* from *meaning* from
*byproduct*, and each seam exists because collapsing it costs something specific.
## Why raw material stays untouched
[raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/raw/CONTRACT.md) keeps a source exactly as it arrived. The reasoning is
[raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/raw/CONTRACT.md) keeps a source exactly as it arrived. The reasoning is
simple once stated: the moment someone "cleans up" or reformats a source on the way in, the
thing later claims get checked against is no longer the thing that was actually said. An
immutable `raw/` means a citation always resolves to the original, not to somebody's tidied
@@ -35,28 +35,39 @@ promotion into `raw/` quietly doubling as the review step it cannot perform.
## Why extraction happens once, through a schema
[types/type-spec.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/types/type-spec.md) is what stands between a raw file and a `kb/` page:
[types/type-spec.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/types/type-spec.md) is what stands between a raw file and a `kb/` page:
a type-spec defines what a conforming instance of a page looks like, and the compiler
(`tools/wikitool`) applies it. The alternative - every query re-reading and re-interpreting the
source on demand - would mean paying the cost of understanding the material every single time,
and getting a slightly different answer each time depending on how the question was phrased.
Extracting once, against a fixed schema, turns "re-read and re-guess" into "look up what was
already compiled." That is the "never re-derive, always compile" principle from
[AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md): understanding a source is expensive and worth doing exactly once,
[AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md): understanding a source is expensive and worth doing exactly once,
after which it becomes a cheap, stable lookup.
## Why a `kb/` page has to stand on its own
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/kb/CONTRACT.md) sets the bar for the compiled layer: a page should answer a
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/kb/CONTRACT.md) sets the bar for the compiled layer: a page should answer a
future question without sending the reader back to the source it came from. That's the payoff
of compiling in the first place - if every answer still bottomed out in "go re-read the raw
file," the `kb/` layer would just be a pointer with extra steps, and the cost of extraction
would have bought nothing. A page that stands alone is what makes the corpus fast and
consistent to query: the work of understanding is already sitting there, done.
The bar is about substance, though, not wording. For a rulebook or a manual the question that
comes back is often about the exact text - what a clause says word for word, which value a table
gives - and a page that carried every such sentence would be the document again, with its own
chance of copying one wrong. So a query may read the raw file for the wording, but only along a
fixed route: from the page that carries the substance to its source page, to the outline in the
derivation's header, and to the one passage's line range. A search over `raw/` would turn the
raw layer back into the thing being queried, which is what compiling was meant to end; reaching
it only through a page keeps every answer anchored in what was already understood. The line
that does not move is that an answer standing in the raw file and on no page still shows a gap
in `kb/` - the query names it rather than making that the normal way to answer.
## Why `reports/` doesn't need to be maintained
[reports/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/reports/CONTRACT.md) treats most of what lands in `reports/` -
[reports/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/reports/CONTRACT.md) treats most of what lands in `reports/` -
lint output, telemetry traces - as disposable. The structural content of a lint report can be
recomputed from the tree at any commit, so keeping an old copy around would just be a second
version of something the tool can already answer on demand, and a second copy is exactly the
@@ -68,7 +79,7 @@ distinction between what's recomputable and what isn't.
## Where `work/` fits
[work/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/work/CONTRACT.md) describes a workshop, not a fifth pipeline stage: a
[work/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/work/CONTRACT.md) describes a workshop, not a fifth pipeline stage: a
place for the notes, extracts and open decisions of a task that spans more than one session, on
its way toward becoming a `kb/` page. It sits beside the raw -> kb -> reports flow rather than
inside it - closer in spirit to a desk than to a conveyor belt.
Explanation-Telemetry-and-evaluation.md
+6 -6
@@ -1,10 +1,10 @@
*Generated from [`docs/explanation/telemetry-and-evaluation.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/telemetry-and-evaluation.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/telemetry-and-evaluation.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/telemetry-and-evaluation.md) at v10.1.0 – edit the repository, not this wiki.*
# Telemetry and evaluation
How the stack observes what an agent did, and how that record turns into a score. This page explains
the design and points at the code. The rules an agent follows live in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/CONTRACT.md) and [reports/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/reports/CONTRACT.md).
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/CONTRACT.md) and [reports/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/reports/CONTRACT.md).
Turning tracing on and reading a session back is a how-to:
[trace-and-score-sessions.md](How-to-Trace-and-score-sessions).
@@ -93,7 +93,7 @@ sides on the same id.
## The trace
One JSON object per line, appended to `reports/telemetry/<session>/trace.jsonl`. The contract is
[tools/chemenu/telemetry/schema.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/chemenu/telemetry/schema.py).
[tools/chemenu/telemetry/schema.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/chemenu/telemetry/schema.py).
| Field | Meaning |
|---|---|
@@ -128,7 +128,7 @@ needs, and charging for a `search` would discourage the one habit that lowers co
## Whether it runs at all
The default depends on how the tree got here, not on one hard-coded switch.
[tools/chemenu/telemetry/policy.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/chemenu/telemetry/policy.py) is the one place that
[tools/chemenu/telemetry/policy.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/chemenu/telemetry/policy.py) is the one place that
resolves it, so `wikitool doctor`, the writer and the MCP server's start-up guard all answer the
same question the same way:
@@ -173,7 +173,7 @@ not on the existing history.
Prompts and assistant replies **are** recorded in cleartext, locally. A failure taxonomy cannot be
read out of hashes, and building one is the first phase of any evaluation work. Four guards make
that defensible, all of them in
[tools/chemenu/telemetry/scrub.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/chemenu/telemetry/scrub.py), so there is one place to
[tools/chemenu/telemetry/scrub.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/chemenu/telemetry/scrub.py), so there is one place to
audit:
1. **Secret scrubbing.** Tokens, keys, `Authorization:` headers and `SECRET=` assignments are
@@ -182,7 +182,7 @@ audit:
3. **A kill switch.** `WIKI_TRACE_CONTENT=0` keeps only `<field>_length` and `<field>_sha256`. The
digest is computed either way, so traces stay comparable.
4. **`raw/` contents never enter a trace at all.** That text is data, not instruction
([AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md) invariant 4), and a trace gets read back later. Callers record a path
([AGENTS.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md) invariant 4), and a trace gets read back later. Callers record a path
and a digest.
Nothing leaves the machine. `reports/` is gitignored, no exporter is configured, and where a vendor
Explanation-Version-model.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/explanation/version-model.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/version-model.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/version-model.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/version-model.md) at v10.1.0 – edit the repository, not this wiki.*
# Why the stack version splits compatibility from migration
Explanation-Why-gates-are-code.md
+4 -4
@@ -1,11 +1,11 @@
*Generated from [`docs/explanation/why-gates-are-code.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/why-gates-are-code.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/why-gates-are-code.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/why-gates-are-code.md) at v10.1.0 – edit the repository, not this wiki.*
# Why gates are code
Chemenu's hard limits - the gates [AGENTS.md § Gates](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#gates) lists - live inside
Chemenu's hard limits - the gates [AGENTS.md § Gates](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#gates) lists - live inside
`tools/wikitool`, not in a paragraph of instructions an agent reads and follows. The rules
themselves, and what to do when one trips, are in [AGENTS.md § Gates](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#gates) and
[instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/gates.md). This page is only about the design choice
themselves, and what to do when one trips, are in [AGENTS.md § Gates](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#gates) and
[instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/gates.md). This page is only about the design choice
underneath them: why code, and why these mechanisms in particular.
<!-- wikitool:toc -->
Explanation.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/explanation/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/explanation/README.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/explanation/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/explanation/README.md) at v10.1.0 – edit the repository, not this wiki.*
# Explanation
Home.md
+6 -6
@@ -1,4 +1,4 @@
*Generated from [`README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/README.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/README.md) at v10.1.0 – edit the repository, not this wiki.*
# Chemenu - Personal IT Knowledge Base
@@ -53,7 +53,7 @@ The human documentation lives under `docs/`, sorted by what you want from it:
- [docs/reference/](Reference) - looking up: commands, exit codes, skills, page
types, configuration. The command reference is
[docs/reference/commands/](Reference-Commands); agents use the same records in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/CONTRACT.md).
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/CONTRACT.md).
- [docs/explanation/](Explanation) - understanding: why the stack is built the way
it is.
@@ -74,7 +74,7 @@ takes a sample document that ships with it through an ingest and a cited answer,
## Changelog
Changes to the wiki stack (schema, skills, `wikitool`, READMEs) are tracked in
[`CHANGES.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/CHANGES.md), not in an inline version history here.
[`CHANGES.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/CHANGES.md), not in an inline version history here.
## Resources
@@ -88,12 +88,12 @@ Chemenu is dual-licensed, because it is two things in one repository.
| Half | Licence | File |
|------|---------|------|
| The stack — `tools/`, `types/` | GNU AGPL-3.0-or-later | [LICENSE](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/LICENSE) |
| The content — `kb/`, `raw/`, `instructions/`, the `CONTRACT.md` layer, and the prose documents at the root | CC-BY-4.0 | [LICENSE-CONTENT](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/LICENSE-CONTENT) |
| The stack — `tools/`, `types/` | GNU AGPL-3.0-or-later | [LICENSE](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/LICENSE) |
| The content — `kb/`, `raw/`, `instructions/`, the `CONTRACT.md` layer, and the prose documents at the root | CC-BY-4.0 | [LICENSE-CONTENT](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/LICENSE-CONTENT) |
The boundary between the two is not a list someone maintains by hand: it is the
file plan `tools/wikitool dist export` already computes, so it cannot drift out
of agreement with what actually ships. [NOTICE](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/NOTICE) states that, and carries
of agreement with what actually ships. [NOTICE](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/NOTICE) states that, and carries
the attribution for the vendored [Commonplace](https://github.com/zby/commonplace)
research base.
How-to-Back-up-to-a-synced-folder.md
+26 -2
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/back-up-to-a-synced-folder.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/back-up-to-a-synced-folder.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/back-up-to-a-synced-folder.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/back-up-to-a-synced-folder.md) at v10.1.0 – edit the repository, not this wiki.*
# Back up to a synced folder
@@ -13,6 +13,7 @@ push to and a copy of the files on this machine that are not in git. What the se
- [Why the instance does not live in the synced folder](#why-the-instance-does-not-live-in-the-synced-folder)
- [Set it up](#set-it-up)
- [One folder for the backup, the intake and the export](#one-folder-for-the-backup-the-intake-and-the-export)
- [What is backed up, and what is not](#what-is-backed-up-and-what-is-not)
- [Restore](#restore)
- [One device writes](#one-device-writes)
@@ -93,6 +94,29 @@ still happened. Fix what the line names - a synced folder that is not there, a f
writable - and run `tools/wikitool backup run`. Another publish does not help: it has nothing new to
push.
## One folder for the backup, the intake and the export
The target may be the one folder your whole setup shares. The backup writes only into `git` and into
the folder of its own machine; everything else in the target belongs to the sync folder, so an
[intake folder](How-to-Collect-from-a-synced-folder) and an
[export folder](How-to-Export-to-a-synced-folder) can sit beside them:
```
<OneDrive - company>\Chemenu\ backup.target
git\ the bare repository (backup init, publish)
<MACHINE>\ this machine's data (backup run, publish)
Inbox\ intake channel, handler folder
Export\ export target
```
The rule behind it: no export and no intake folder may be, or lie in, `<target>\git` or a machine
folder. For the export that holds from above as well (its path must not contain them either); an
intake folder may sit above them, since `upload collect` reads only the top level of its folder.
`doctor` checks the part that the settings can show. What they cannot show - the folder of another
machine, or one that a machine starts to fill later - is checked when the command runs: `upload
collect` and `export run` refuse a folder that holds or lies in a `.chemenu-backup.json`, a bare
repository or (for `upload collect`) a `.wikitool-export.json`, and touch nothing in it.
## What is backed up, and what is not
| Path | Backed up | Why |
@@ -142,7 +166,7 @@ On the same machine or a new one:
> Bootstrap this checkout.
It follows [instructions/bootstrap.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/bootstrap.md): the preflight, the
It follows [instructions/bootstrap.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/bootstrap.md): the preflight, the
published skills, `doctor`.
4. **Provide the secrets again.** They were never in the backup. Put each value back where its
reference reads it - 1Password, the secrets file, or a `WIKITOOL_SECRET_<NAME>` variable
How-to-Browse-with-obsidian.md
+3 -3
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/browse-with-obsidian.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/browse-with-obsidian.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/browse-with-obsidian.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/browse-with-obsidian.md) at v10.1.0 – edit the repository, not this wiki.*
# Browse the wiki in Obsidian
@@ -25,7 +25,7 @@ keeps consistent across the whole wiki. A hand edit can quietly break them. Rena
file in Obsidian is the clearest case: a title is the page's only identifier, so it also appears in
every wikilink and citation that points at the page. A rename goes through
`tools/wikitool rename`, which updates every reference. The full procedure is
[instructions/page-lifecycle.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/page-lifecycle.md).
[instructions/page-lifecycle.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/page-lifecycle.md).
Generated files - `kb/index.md`, `kb/log.md`, `kb/provenance.md` and every `INDEX.md` - are
rebuilt by the tool and never edited by hand.
@@ -45,7 +45,7 @@ Knowing the naming rules makes the vault easier to find your way in:
page's title by case alone. A file's whole path below the instance root stays at 160 characters
or fewer, so a Windows checkout works without long paths. `wikitool new` and `wikitool rename`
refuse titles that break these rules, and `wikitool lint` reports existing ones.
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/kb/CONTRACT.md) has the full rule.
[kb/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/kb/CONTRACT.md) has the full rule.
## Letting the structure grow
How-to-Capture-repository-docs.md
+3 -2
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/capture-repository-docs.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/capture-repository-docs.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/capture-repository-docs.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/capture-repository-docs.md) at v10.1.0 – edit the repository, not this wiki.*
# Capture documentation from a git repository
@@ -45,7 +45,8 @@ Update the captured repositories
```
The agent checks every captured bundle. It takes one changed bundle per run into the wiki, replacing
the old edition as a whole, and tells you how many are still behind. Run it again for the next one.
the old edition as a whole, and tells you how many are still behind. For the next one, `/clear` and
say it again - each bundle is a run of its own in a fresh context.
`raw capture --update` and `raw accept --replaces-bundle` are the commands underneath. What each one
does is in the [command reference](Reference-Commands-Raw-material-and-uploads).
How-to-Collect-from-a-synced-folder.md
+15 -3
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/collect-from-a-synced-folder.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/collect-from-a-synced-folder.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/collect-from-a-synced-folder.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/collect-from-a-synced-folder.md) at v10.1.0 – edit the repository, not this wiki.*
# Collect documents from a synced folder
@@ -46,8 +46,13 @@ checkout, so it is yours to edit by hand in `.wikitool-local.json`.
}
```
`path` has to be absolute and lie outside the instance. Check it with `tools/wikitool doctor`, then
pull:
`path` has to be absolute and lie outside the instance. It may sit in the folder that also holds
the backup and an export, as `<OneDrive - company>\Chemenu\Inbox` beside `git`, the machine folder
and `Export` ([the layout](How-to-Back-up-to-a-synced-folder#one-folder-for-the-backup-the-intake-and-the-export)),
but it must not be, or lie in, `<backup.target>/git` or a machine folder of the backup. `upload
collect` also refuses, for that channel alone and before it reads anything, a folder that holds or
lies in a machine folder of any machine, a bare repository or an export: it empties its folder, and
these are not submissions. Check it with `tools/wikitool doctor`, then pull:
```bash
tools/wikitool upload collect # every pull channel
@@ -86,6 +91,13 @@ a client that pauses for longer than the interval in the middle of a file and th
file is stored incomplete, and the review is where you notice it. Raise the value if your client
works over a slow link.
OneDrive on Windows, checked with 10.0.0: a file uploaded from a phone shows up in the folder as an
online-only placeholder (attributes `Archive, ReparsePoint`) that keeps its original creation and
modification times - a week old in the check, not the time it arrived. So `settle_seconds` does not
hold such a file back; reading it, which `collect` does, downloads it, and the size-and-sha256
check is what guards the copy. The copy in the quarantine carries the time of the collect, not the
original one.
## Deleting in a synced folder deletes everywhere
A file removed from a folder that OneDrive syncs is removed on every device that syncs it. That is
How-to-Export-guidelines.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/export-guidelines.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/export-guidelines.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/export-guidelines.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/export-guidelines.md) at v10.1.0 – edit the repository, not this wiki.*
# Export guidelines to captured repositories
How-to-Export-to-a-synced-folder.md
+44 -17
@@ -1,10 +1,10 @@
*Generated from [`docs/how-to/export-to-a-synced-folder.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/export-to-a-synced-folder.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/export-to-a-synced-folder.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/export-to-a-synced-folder.md) at v10.1.0 – edit the repository, not this wiki.*
# Export to a synced folder
Give a reader outside the repository a read-only version of part of `kb/`: the selected pages,
written as files into a folder - typically one a sync client such as OneDrive uploads, so that a
Microsoft 365 tenant can index them. The export is derived. It is regenerated from the committed
Give a reader outside the repository a read-only version of part or all of `kb/`: the selected
pages, written as files into a folder - typically one a sync client such as OneDrive uploads, so
that a Microsoft 365 tenant can index them. The export is derived. It is regenerated from the committed
pages on every run, nobody edits it, and nothing in it flows back into the wiki. What the settings
look like: [configuration reference](Reference-Configuration#export-to-a-folder).
@@ -22,25 +22,37 @@ look like: [configuration reference](Reference-Configuration#export-to-a-folder)
## Pick the pages and the format
- **Which pages.** Only what a target's `include` selects is written; there is no "everything". A
selection is a list of `search` predicates that must all hold - `tags=shared`, `kind=system`,
`!draft` - and a page that fits any selection of the target is exported. Try a selection with
`tools/wikitool search --field tags=shared` first.
- **Which pages.** Only what a target's `include` selects is written. `"include": "all"` selects
every page, also one that lies directly under `kb/`, and no generated file. Otherwise a selection
is a list of `search` predicates that must all hold - `tags=shared`, `kind=system`, `!draft` - and
a page that fits any selection of the target is exported: `"include": [["tags=shared"]]`. Try a
selection with `tools/wikitool search --field tags=shared` first. A missing or empty `include` is
an error, not "all". With `"all"`, every page you add later goes into the export too - on a target
with `after_publish` at the next `publish`.
- **An entry point.** The folder holds the pages only. `"index": true` adds `index.md` in the root
of the folder and an `INDEX.md` per collection, so someone who opens the folder finds the
collections, their areas and the pages with their summaries by following links. The files list what
the target exports and nothing else, and the link text is the page's title.
- **Which format.** Two ways, equally valid:
- `markdown` writes the pages as `.md` files. It needs nothing but the folder.
- `command` hands those files to a converter of your own and exports what it produces - Word
documents with `pandoc`, say.
Whether a Microsoft 365 tenant indexes `.md` files for Copilot is not settled for every tenant.
Try a small `markdown` target first; if the tenant does not pick the files up, the same pages go
through a `command` target as a format it does.
In the one tenant checked so far (10.0.0), Microsoft 365 Copilot found the `.md` files of a
`markdown` target in OneDrive and used them. Tenants differ, so start with a small `markdown`
target; if yours does not pick the files up, the same pages go through a `command` target as a
format it does.
## Set it up
1. **Pick the folder**: a folder inside the synced folder, for example
`C:\Users\<you>\OneDrive - <company>\Chemenu-Lesefassung`. It must be empty or not exist yet,
and it must lie outside the instance, outside every other export target, outside every `intake`
folder and outside `backup.target`.
and it must lie outside the instance, outside every other export target and every `intake`
folder, and apart from the folders the backup writes into: it may not be, lie in or contain
`<backup.target>\git` or the backup's machine folder, nor lie in another machine's folder or a
bare repository. A folder beside them is fine, so one synced folder can hold the backup, an
intake folder and the export, with the export in `<OneDrive - company>\Chemenu\Export`
([the layout](How-to-Back-up-to-a-synced-folder#one-folder-for-the-backup-the-intake-and-the-export)).
2. **Write the `export` section** into `.wikitool-local.json` at the root of the instance - by hand,
since its fields decide what leaves this machine and which program runs, and `config set`
refuses them:
@@ -66,6 +78,10 @@ look like: [configuration reference](Reference-Configuration#export-to-a-folder)
(the default) the title, type, summary and modified date, `full` the page's whole frontmatter.
`links: relative` (the default) turns a link to an exported page into a relative Markdown link;
`links: text` leaves the page's name. A link to a page that is not exported is always plain text.
The index files link relatively whatever `links` says.
To put the whole of `kb/` into the folder with an entry point, write `"include": "all"` and
`"index": true` instead of the `include` selection above.
3. **Tell the agent:**
> Export the pages.
@@ -77,12 +93,12 @@ look like: [configuration reference](Reference-Configuration#export-to-a-folder)
what is wrong with the section.
The export reads the committed pages. A `kb/` or `types/` with uncommitted changes is refused, so
every file names the commit its pages come from. Publish first.
every file names the commit its page comes from. Publish first.
## What a run does to the folder
The folder holds the pages and one file of the stack's own, `.wikitool-export.json`, which lists
what the last run wrote.
The folder holds the pages, the index files if the target asks for them, and one file of the stack's
own, `.wikitool-export.json`, which lists what the last run wrote.
- **Only its own files.** A run removes a file only when it is listed and no longer exported, and a
folder only when the run created it. Anything else in the folder stays as it is.
@@ -94,6 +110,16 @@ what the last run wrote.
- **An empty selection is an error.** A run that finds no page does not delete the earlier export.
- **An unchanged export stays untouched.** A run that has nothing to change writes nothing, not even
a modification time - the sync client has nothing to upload.
- **A commit rewrites the files of the pages it touched.** Each file names the last commit of its own
page, so an ingest that changed three pages rewrites three files, however large the selection.
A file of a page the commit did not touch changes only when its text does: a link to a page that
has just entered the selection becomes a relative link, one to a page that left it plain text.
The first run after updating to a version that brought this rewrites every file once.
- **An index follows the pages.** A collection's `INDEX.md` changes when one of its pages does, and
`index.md`, which lists them all, with every commit to an exported page. Setting `index` back to
`false` removes the index files at the next run and leaves every other file as it is. A page
`index.md` in a collection, next to the generated `INDEX.md`, stops the target: a synced folder on
Windows would make them one file.
- **A growing selection needs no confirmation.** New pages are reported as new; read the list
before the sync client uploads them.
@@ -142,7 +168,8 @@ program's stderr. The staging directory is always removed.
## What does not come back in
An export is never a source. Every Markdown file starts with a frontmatter block whose first key is
`wikitool_export`, and `tools/wikitool raw accept` and `raw capture` refuse a file that carries it.
`wikitool_export` - the index files too - and `tools/wikitool raw accept` and `raw capture` refuse a
file that carries it.
A converter may lose that key, so `raw accept` also refuses any file in `incoming/` that is byte
for byte one of the files in the manifest of a configured target - and warns, without refusing,
when a target has no manifest yet or the `export` section is broken.
How-to-Fetch-pages-behind-a-login.md
+2 -2
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/fetch-pages-behind-a-login.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/fetch-pages-behind-a-login.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/fetch-pages-behind-a-login.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/fetch-pages-behind-a-login.md) at v10.1.0 – edit the repository, not this wiki.*
# Fetch pages behind a login
@@ -61,7 +61,7 @@ it ([Write a site handler](How-to-Write-a-site-handler)):
`signed_in` is something only a signed-in page carries - a logout link, an account menu.
`signed_out` is something only a page that was not carries - the teaser box that asks you to
subscribe. One of the two is enough; open a saved article and a saved teaser and pick an element
that tells them apart. [sites/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/sites/CONTRACT.md) has the exact format.
that tells them apart. [sites/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/sites/CONTRACT.md) has the exact format.
## 3. Route the website through your session
How-to-Ingest-a-source.md
+31 -5
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/ingest-a-source.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/ingest-a-source.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/ingest-a-source.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/ingest-a-source.md) at v10.1.0 – edit the repository, not this wiki.*
# Ingest a source
@@ -47,7 +47,14 @@ pick a directory:
folder is accepted whole; the agent runs this itself when it meets such a file. An Excel file's
sheets land beside the text as tab-separated files, with their formulas listed apart. A ZIP is
unpacked into a folder instead, and a mail's attachments land beside it. A scanned PDF has no
text to derive - the tool warns, and there is no OCR.
text to derive - the tool warns, and there is no OCR. A PDF with an outline gets it in the
header of the text, each entry with its page and the line where that page starts; struck-through
text, highlighted, bold or clearly differently coloured text, and italic text are marked in the
text (`~~…~~`, `**…**`, `*…*`) with a legend in the header, and what a colour means is for the
reader to take from the document itself. The tool marks sparingly, in running text and
not in code or tables. Pages with pictures or drawings that carry content are named in a `CHECK`
line and in the header of the text (`images`), because the pictures are not in the text; the
pages whose text could not be matched to the layout are named there too (`unmatched`).
- **A page behind a paywall or a login** is saved from your browser into `incoming/`, HTML only. The
agent derives the same text from that file with `raw fetch --html`. For a website you read often,
sign in once instead and let the wiki fetch it with your session:
@@ -69,12 +76,31 @@ Ingest
A bare `Ingest` takes the oldest entry waiting in `incoming/`. `tools/wikitool raw pending` lists
what is waiting and names the entry the next bare `Ingest` will take.
One run takes one entry. When more are waiting, the agent says how many and ends with `/clear`,
then `ingest`: the next source belongs in a fresh context, because the previous one would otherwise
still be in view while its facts and citations are written for the next, and the iteration budget
fits about two runs. The agent does not start the next entry itself; if you ask it to go on in the
same context, it names that reason once and then does.
## Where the agent asks you
- **Before promoting the file into `raw/`**, the agent asks how faithful the capture is
(`fidelity`) and what the material may claim about its subject (`authority`). Both are recorded
once and never guessed: you know the answers now, and nobody can recover them later.
[raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/raw/CONTRACT.md) defines the values.
[raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/raw/CONTRACT.md) defines the values.
- **Before looking at page images**, when a PDF names more than 20 pages the text does not hold.
The agent reads up to 20 such pages on its own, through a subagent where the harness has one,
and records what it saw on the source page, with the page number and the note that it was read
from the image. For more pages it reads none first: it tells you the count and a token
estimate, proposes a selection, and you choose all, none or a selection. Pages nobody looked at
are listed with the reason. Only whole pages are read; a harness that cannot show PDF pages as
images gets the page numbers named instead.
- **Before reading a long document in parts.** When the text `raw derive` or `raw fetch` wrote
is longer than one pass reads well, the tool says so in a `CHECK` line, and `raw pending` marks
a long text file the same way. The agent then does not read it in one go: it cuts it into parts
along its outline (or its pages), shows you the plan in a workshop under `work/`, and after your
answer reads one part at a time, noting each fact with where it stands. The document stays one
file in `raw/` and gets one source page, written from those notes.
- **Before writing pages**, the agent presents the key takeaways and asks which points matter most
and what to emphasise. The pages follow your answer.
- **When it publishes**, a source that touches ten or more files stops at the Mass-Update Gate
@@ -108,7 +134,7 @@ tools/wikitool upload show <id> # one submission's manifest: who sent it,
```
Ask the agent to review the queue, and it follows
[instructions/ingest-queue.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/ingest-queue.md). Accepting a submission stops at
[instructions/ingest-queue.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/ingest-queue.md). Accepting a submission stops at
the Upload Review Gate (exit 42), which shows you the manifest and the `--confirm` line. Once you
approve, the file is in `incoming/`, and from there it is ingested like any other. Rejecting it needs
no clearance. The file is deleted, and its checksum and your reason stay in the ledger.
@@ -117,5 +143,5 @@ no clearance. The file is deleted, and its checksum and your reason stay in the
- [Capture documentation from a repository](How-to-Capture-repository-docs) - for material that lives
in git and will change.
- [raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/raw/CONTRACT.md) - what belongs in `raw/`, and why its content is data,
- [raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/raw/CONTRACT.md) - what belongs in `raw/`, and why its content is data,
never instructions.
How-to-Maintain-the-wiki.md
+2 -2
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/maintain-the-wiki.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/maintain-the-wiki.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/maintain-the-wiki.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/maintain-the-wiki.md) at v10.1.0 – edit the repository, not this wiki.*
# Maintain the wiki
@@ -15,7 +15,7 @@ Lint the wiki
Do this periodically. The agent suggests it on its own every ten sources. The full schedule - which
task runs how often, and with which command - is the
[maintenance schedule](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/CONTRACT.md#maintenance-schedule) in `tools/CONTRACT.md`, next
[maintenance schedule](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/CONTRACT.md#maintenance-schedule) in `tools/CONTRACT.md`, next
to the command records it depends on.
## What you get back
How-to-Query-the-wiki.md
+26 -2
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/query-the-wiki.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/query-the-wiki.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/query-the-wiki.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/query-the-wiki.md) at v10.1.0 – edit the repository, not this wiki.*
# Query the wiki
@@ -27,6 +27,28 @@ If an answer is worth keeping - a comparison you will want again, a synthesis ac
the agent proposes filing it back as a page. It names the pages it would create before it creates
any, and nothing is written unless you agree.
## Asking for the exact wording
Some questions are about the text itself - what a rule says word for word, which value stands in
a table of a manual:
```text
What exactly does rule 16:3e say?
Which torque does table 4 of the assembly manual give?
```
The agent still starts from a page. For the wording it follows that page to its source page, then
to the outline in the header of the source's derived text, and reads only the passage's lines - it
never searches `raw/`. The quote comes in the language of the original, with the source page and
the page number. Markings the derivation added (bold, italic, highlighting) are left out of the
quote, and struck-through text is said to be struck rather than quoted as if it were in force.
Two answers point at work for later. If the derived text has no outline and is too long to read
whole, the agent says the wording cannot be reached directly and names the way to change that: a
new `raw derive` of the original, accepted with `raw accept --replaces`. If the answer stands in
the source but on no page, the agent answers with the quote and names the page that is missing it,
offering to complete it - it writes nothing on its own.
## Searching yourself
The agent searches with the same command you can run:
@@ -49,4 +71,6 @@ a graph, see [Browse the wiki in Obsidian](How-to-Browse-with-obsidian).
## Asking from another machine
A consumer that is not a terminal on this machine can ask the same questions through the MCP read
server and get the same answers: [Set up the MCP read server](How-to-Set-up-the-mcp-server).
server and get the same answers: [Set up the MCP read server](How-to-Set-up-the-mcp-server). The one
difference is the exact wording above: the read server serves pages only, never `raw/`, so a
reader over MCP gets the wording only where a page carries it.
How-to-Run-the-weekly-review.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/run-the-weekly-review.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/run-the-weekly-review.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/run-the-weekly-review.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/run-the-weekly-review.md) at v10.1.0 – edit the repository, not this wiki.*
# Run the weekly review
How-to-Set-up-the-mcp-server.md
+2 -2
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/set-up-the-mcp-server.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/set-up-the-mcp-server.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/set-up-the-mcp-server.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/set-up-the-mcp-server.md) at v10.1.0 – edit the repository, not this wiki.*
# Set up the MCP read server
@@ -7,7 +7,7 @@ machine - another agent, a chat client, a service - can ask it the same question
answers. This page covers what you set up around the server: the client, the authentication in
front of it, the optional `submit` path, and how to check that it works. Installing, starting and
syncing the server itself is
[instructions/mcp-read-server.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/mcp-read-server.md), which an agent follows
[instructions/mcp-read-server.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/mcp-read-server.md), which an agent follows
and which is readable for you as well.
<!-- wikitool:toc -->
How-to-Trace-and-score-sessions.md
+6 -6
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/trace-and-score-sessions.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/trace-and-score-sessions.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/trace-and-score-sessions.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/trace-and-score-sessions.md) at v10.1.0 – edit the repository, not this wiki.*
# Trace and score agent sessions
@@ -40,7 +40,7 @@ yourself keeps one task in one trace:
export WIKITOOL_SESSION_ID="my-task"
```
[instructions/session-setup.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/session-setup.md) has the line for each shell,
[instructions/session-setup.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/session-setup.md) has the line for each shell,
and what to do on a harness that starts a fresh shell for every command.
## Reading a session back
@@ -110,7 +110,7 @@ guarantee: an agent that reorders the flags gets around it.
### Copilot CLI
[.github/hooks/wiki-trace.json](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/.github/hooks/wiki-trace.json) is committed, so a clone
[.github/hooks/wiki-trace.json](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/.github/hooks/wiki-trace.json) is committed, so a clone
brings its own tracing. Eleven events go to `tools/trace_ingest.py`. `disableAllHooks` in
`.github/copilot/settings.json` turns them off without deleting anything. `userPromptSubmitted` is
among them, so `clearance-ended-the-turn` can be scored on Copilot CLI with no change.
@@ -146,7 +146,7 @@ hooks:
VS Code Chat has no hooks, so nothing observes a session while it runs. Its chronicle store
(`session-store.db`, the same schema as Copilot CLI's) keeps sessions, turns and touched files, and
[tools/import_chronicle.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/import_chronicle.py) rebuilds a trace from it afterwards:
[tools/import_chronicle.py](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/import_chronicle.py) rebuilds a trace from it afterwards:
```bash
tools/import_chronicle.py --dry-run # this repository's sessions
@@ -160,8 +160,8 @@ that a file was touched, not that a tool was about to run.
### Mistral Vibe
[.vibe/hooks.toml](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/.vibe/hooks.toml) declares the three hooks Vibe has, and
[.vibe/config.toml](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/.vibe/config.toml) keeps the telemetry settings in the repository rather
[.vibe/hooks.toml](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/.vibe/hooks.toml) declares the three hooks Vibe has, and
[.vibe/config.toml](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/.vibe/config.toml) keeps the telemetry settings in the repository rather
than in someone's shell profile. Both were checked against the installed CLI's own loader
(`mistral-vibe 2.24.2`) rather than against its documentation. What that turned up:
How-to-Troubleshoot.md
+26 -5
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/troubleshoot.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/troubleshoot.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/troubleshoot.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/troubleshoot.md) at v10.1.0 – edit the repository, not this wiki.*
# Troubleshoot an instance
@@ -9,6 +9,20 @@ names its own stops - what is missing, why, and the command that fixes it - and
For the MCP read server, see the troubleshooting section of
[Set up the MCP read server](How-to-Set-up-the-mcp-server#troubleshooting).
<!-- wikitool:toc -->
## Contents
- [`tools/wikitool` ends with `STOP - this checkout is not set up yet` (exit 42)](#toolswikitool-ends-with-stop---this-checkout-is-not-set-up-yet-exit-42)
- [The agent offers no skills (`wiki-ingest`, `wiki-query`, ...)](#the-agent-offers-no-skills-wiki-ingest-wiki-query)
- [`doctor` reports `personalization: FAIL`](#doctor-reports-personalization-fail)
- [`doctor` reports `environment: WARN`](#doctor-reports-environment-warn)
- [`doctor` reports `telemetry: WARN`](#doctor-reports-telemetry-warn)
- [`new` stops with "No author configured"](#new-stops-with-no-author-configured)
- [`publish` ends with exit 42 (Mass-Update Gate)](#publish-ends-with-exit-42-mass-update-gate)
- [`publish` ends with exit 42 (Publish-Remote Gate)](#publish-ends-with-exit-42-publish-remote-gate)
- [Setup or an update fails, and you want to report it](#setup-or-an-update-fails-and-you-want-to-report-it)
<!-- /wikitool:toc -->
## `tools/wikitool` ends with `STOP - this checkout is not set up yet` (exit 42)
The preflight has not run in this checkout yet, or not since the last update. Run it:
@@ -20,7 +34,7 @@ tools/preflight.sh
In PowerShell 7 on Windows, the call is
`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1`. The `-ExecutionPolicy Bypass`
applies to that one process only and changes no setting.
[instructions/preflight.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/preflight.md) is the procedure, and its output says
[instructions/preflight.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/preflight.md) is the procedure, and its output says
what to install if anything is missing.
## The agent offers no skills (`wiki-ingest`, `wiki-query`, ...)
@@ -37,7 +51,7 @@ Then restart the agent session. A harness reads its skills only when the session
`USER.md` or `SOUL.md` is missing, or still carries the sentinel line from its template. A renamed
template does not count as a filled one. Ask the agent to run the personalization step of
[instructions/setup-instance.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/setup-instance.md). For a further clone of an
[instructions/setup-instance.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/setup-instance.md). For a further clone of an
older instance, that is the only step to catch up on.
## `doctor` reports `environment: WARN`
@@ -65,7 +79,7 @@ export `WIKI_AUTHOR` ([configuration](Reference-Configuration)).
Expected when ten or more files are counted, for example on the very first commit of a new instance
or after a lint pass. It is not an error. It asks that a human see the output: the complete file
list and the exact `--confirm <token>` line that publishes once you approve.
[instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/gates.md) has the details.
[instructions/gates.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/gates.md) has the details.
## `publish` ends with exit 42 (Publish-Remote Gate)
@@ -77,7 +91,7 @@ opening a gate on its own initiative, which it must never do.
## Setup or an update fails, and you want to report it
Ask the agent to follow [instructions/bug-report.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/bug-report.md), or call
Ask the agent to follow [instructions/bug-report.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/bug-report.md), or call
`tools/bugreport` yourself. It works the same from Bash, Git Bash and PowerShell. It finds a
Python 3.8 or newer by itself, skipping the Store aliases `python3` points at on Windows. It needs
neither the preflight nor a working `wikitool`, and if it finds no Python, it says why.
@@ -96,4 +110,11 @@ The script writes a bundle to `reports/bugreport-<timestamp>/`, with a zip besid
the checklist and the candidates file contain the originals, and they sit beside the bundle,
never in it.
- **Correcting the chronology afterwards:** if the chronology turns out wrong or missing once the
bundle exists, the agent swaps it with `tools/bugreport --bundle … --chronology …` instead of
collecting again. That changes only `CHRONOLOGY.md` and the zip - no `wikitool` command runs
again and no second bundle appears - and the new text is scrubbed, kept clear of titles and
pseudonymised like the rest. After a stage 2 run the agent reads the new chronology and runs
stage 2 once more for names it finds there.
Nothing is uploaded anywhere. You choose the channel.
How-to-Update-an-instance.md
+3 -3
@@ -1,10 +1,10 @@
*Generated from [`docs/how-to/update-an-instance.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/update-an-instance.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/update-an-instance.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/update-an-instance.md) at v10.1.0 – edit the repository, not this wiki.*
# Update an instance
Bring an instance installed from a release up to a newer release of the stack: the tools, types,
instructions and contracts. Your pages stay yours. The agent carries out the upgrade by following
[instructions/upgrade-instance.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/upgrade-instance.md), which sets the order of
[instructions/upgrade-instance.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/upgrade-instance.md), which sets the order of
the steps, what each one decides, and where the session has to restart. This page covers the
decision before it, and what you'll see along the way.
@@ -102,7 +102,7 @@ carry it. After another clone has pulled the new release, run `tools/wikitool co
there too. Until then `doctor` reports each leftover file as a `FAIL`, and a command that needs
one of those settings stops with exit 42 and names that command. `migrate done` is still recorded
only once per instance
([10.0.0-settings-tiers.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/migrations/10.0.0-settings-tiers.md)).
([10.0.0-settings-tiers.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/migrations/10.0.0-settings-tiers.md)).
If a step fails for a reason that isn't obvious, the agent stops and offers you a bug report rather
than improvising ([Troubleshoot](How-to-Troubleshoot)).
How-to-Use-1password-for-settings.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/use-1password-for-settings.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/use-1password-for-settings.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/use-1password-for-settings.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/use-1password-for-settings.md) at v10.1.0 – edit the repository, not this wiki.*
# Use 1Password for settings
How-to-Write-a-site-handler.md
+2 -2
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/write-a-site-handler.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/write-a-site-handler.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/write-a-site-handler.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/write-a-site-handler.md) at v10.1.0 – edit the repository, not this wiki.*
# Write a site handler
@@ -48,7 +48,7 @@ A small handler that drops a cookie banner and a footer on one host:
`host` is required and matched exactly, in lower case. `path` narrows the handler to a part of the
website; among the handlers of a host the longest matching path wins. The other fields - `rewrites`,
`checks`, `parent` - and what each selector may say are in [sites/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/sites/CONTRACT.md).
`checks`, `parent` - and what each selector may say are in [sites/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/sites/CONTRACT.md).
A page that is an index - a module overview, a table of contents - can name its subpages, so that
`raw fetch <url> --scope` fetches it and them as one bundle in `incoming/<name>/`:
How-to.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/how-to/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/how-to/README.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/how-to/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/how-to/README.md) at v10.1.0 – edit the repository, not this wiki.*
# How-to guides
Install.md
+2 -2
@@ -1,4 +1,4 @@
*Generated from [`INSTALL.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/INSTALL.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`INSTALL.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/INSTALL.md) at v10.1.0 – edit the repository, not this wiki.*
# Install a Chemenu instance
@@ -7,7 +7,7 @@ way to get one: an agent installs the **latest release** into an empty folder yo
agent runs the steps, following `instructions/setup-instance.md` from that same release. This
page covers what you prepare beforehand, the sentence you give the agent, what it asks you, and
what to do when it stops. The full command reference is in
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/tools/CONTRACT.md).
[tools/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/tools/CONTRACT.md).
The optional **MCP read server** is set up separately, once the instance runs:
[docs/how-to/set-up-the-mcp-server.md](How-to-Set-up-the-mcp-server). It serves the same
Reference-Commands-Catalog-and-log.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/catalog-and-log.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/catalog-and-log.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/catalog-and-log.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/catalog-and-log.md) at v10.1.0 – edit the repository, not this wiki.*
# Catalog and log
Reference-Commands-Configuration.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/configuration.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/configuration.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/configuration.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/configuration.md) at v10.1.0 – edit the repository, not this wiki.*
# Configuration
Reference-Commands-Content-migrations.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/content-migrations.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/content-migrations.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/content-migrations.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/content-migrations.md) at v10.1.0 – edit the repository, not this wiki.*
# Content migrations
Reference-Commands-Distribution-and-versioning.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/distribution-and-versioning.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/distribution-and-versioning.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/distribution-and-versioning.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/distribution-and-versioning.md) at v10.1.0 – edit the repository, not this wiki.*
# Distribution and versioning
Reference-Commands-Exports.md
+7 -6
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/exports.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/exports.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/exports.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/exports.md) at v10.1.0 – edit the repository, not this wiki.*
# Exports
@@ -111,11 +111,12 @@ tools/wikitool export run work-notes
**Behaviour**
- An export is derived: regenerated from `kb/` on every run, never edited, never read back into the wiki. Each target names its pages with an allowlist of `search --field` predicates (`include`: a list of selections, AND within one, OR across them), a handler (`markdown`, or `command` for the operator's converter) and the folder (`path`, outside the instance). Nothing is exported that no selection names.
- Every Markdown file starts with a frontmatter block whose `wikitool_export` line says `kind=pages instance=<origin URL without userinfo, or local> commit=<newest commit touching a selected page>`, followed by `title`, `type`, `summary`, `modified` where the page has them (`frontmatter: summary`, the default; `minimal` keeps only the marker, `full` the whole frontmatter), then the page text. A file keeps its path below `kb/`. A wikilink to an exported page becomes a relative Markdown link (`links: relative`, the default) or plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote markers stay; code is never rewritten. LF endings, no timestamp: the same pages give the same bytes.
- An export is derived: regenerated from `kb/` on every run, never edited, never read back into the wiki. Each target names its pages with `include`: either the string `"all"` (every page `search` knows, also one directly under `kb/`; never a generated or meta file) or an allowlist of `search --field` predicates (a list of selections, AND within one, OR across them), a handler (`markdown`, or `command` for the operator's converter) and the folder (`path`, outside the instance). A missing or empty `include` is an error, never "all": nothing is exported that the target does not name.
- With `index: true` a target also gets `index.md` in its root and an `INDEX.md` per collection (and per area the catalog gives a file of its own), built from the exported pages alone: collections and areas with their counts, then tables of page, type, summary and last modified, with relative Markdown links whatever `links` says. No date of the day and no hint for an agent; the marker is `kind=index` with the newest commit of the pages the file lists, so a commit to one collection rewrites that collection's `INDEX.md` and `index.md` only. The files are ordinary files of the run (manifest, atomic write, removed once no longer generated); the `command` handler receives them with the pages. A page `<collection>/index.md` beside the generated `INDEX.md` stops the target.
- Every Markdown file starts with a frontmatter block whose `wikitool_export` line says `kind=pages instance=<origin URL without userinfo, or local> commit=<newest commit touching this page>`, followed by `title`, `type`, `summary`, `modified` where the page has them (`frontmatter: summary`, the default; `minimal` keeps only the marker, `full` the whole frontmatter), then the page text. A file keeps its path below `kb/`. A wikilink to an exported page becomes a relative Markdown link (`links: relative`, the default) or plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote markers stay; code is never rewritten. LF endings, no timestamp: the same pages give the same bytes, and a commit that touches one page rewrites that page's file alone - plus any file whose links change because a page entered or left the selection. The manifest's `commit` is the newest commit touching any selected page; all of them come from one `git log`, however many pages are selected.
- Handler `command`: the Markdown is rendered into a staging directory, the program of `export.commands` runs with no shell, stdin closed and a timeout of 300 s, and receives the Markdown directory and an empty output directory as its last two arguments. On exit 0 the regular files in the output directory (recursively) become the target's content; any other end leaves the target unchanged and prints the end of its stderr. The staging directory is always removed.
- The folder is only ever touched by what it holds in its manifest, `.wikitool-export.json`: the files an earlier run wrote, each with its sha256. A run removes only manifest files that are no longer exported (and folders it created, once empty), rewrites a manifest file that is missing, leaves every other file byte for byte as it was, and writes nothing - not even a modification time - for a file that did not change. A run that changes nothing writes nothing at all.
- Refuses a target, before writing anything into it: a non-empty folder with no manifest (it is somebody's, not an export), a manifest file that was changed in the folder since the last run (naming it), a foreign file where an exported one belongs, an empty selection (it would delete the earlier export), an unknown field in a predicate. A missing folder is created.
- Refuses a target, before writing anything into it: a non-empty folder with no manifest (it is somebody's, not an export), a folder that holds or lies in backup data (a `.chemenu-backup.json` - any machine's, also one this checkout does not know - or a bare git repository), a manifest file that was changed in the folder since the last run (naming it), a foreign file where an exported one belongs, an empty selection (it would delete the earlier export), an unknown field in a predicate. A missing folder is created.
- Refuses everything, before any target: uncommitted changes in `kb/` or `types/` (untracked files included), no git checkout, a page under `kb/` with unreadable frontmatter, an unknown target name, a broken `export` section.
- Output per target: `new` (with the page titles), `changed`, `removed`, the count `unchanged`, or `failed` with the reason. A page that newly enters a selection is reported as new, not gated. Exit 0 when every target ran; exit 1 when any failed - the others were still served.
- A target with `after_publish: true` is also refreshed by every `publish` that committed, `--no-push` included; a failure there is a warning, the commit stands, and this command is the repeat.
@@ -132,9 +133,9 @@ tools/wikitool export run work-notes
**Exit codes**
- `0` - success
- `1` - No `export` section, an unknown target name, or a broken section (a predicate that does not parse, a path inside the instance or in the way of another target, an `intake` folder or the backup target)
- `1` - No `export` section, an unknown target name, or a broken section (a predicate that does not parse, a path inside the instance or in the way of another target, an `intake` folder or a folder the backup writes into: `<backup.target>/git` or this machine's folder)
- `1` - `kb/` or `types/` has uncommitted changes, the instance is not a git checkout, or a page under `kb/` has unreadable frontmatter
- `1` - A target failed: a foreign folder, a file changed in the target, an empty selection, an unknown field, a converter that did not exit 0
- `1` - A target failed: a foreign folder, a folder in backup data, a file changed in the target, an empty selection, an unknown field, two files that differ only in case (also a page and a generated index file), a converter that did not exit 0
**See also**
Reference-Commands-Finding-and-checking.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/finding-and-checking.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/finding-and-checking.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/finding-and-checking.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/finding-and-checking.md) at v10.1.0 – edit the repository, not this wiki.*
# Finding and checking
Reference-Commands-Git.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/git.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/git.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/git.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/git.md) at v10.1.0 – edit the repository, not this wiki.*
# Git
Reference-Commands-Instance-health.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/instance-health.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/instance-health.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/instance-health.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/instance-health.md) at v10.1.0 – edit the repository, not this wiki.*
# Instance health
Reference-Commands-Links-and-citations.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/links-and-citations.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/links-and-citations.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/links-and-citations.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/links-and-citations.md) at v10.1.0 – edit the repository, not this wiki.*
# Links and citations
Reference-Commands-Pages.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/pages.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/pages.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/pages.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/pages.md) at v10.1.0 – edit the repository, not this wiki.*
# Pages
Reference-Commands-Provenance.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/provenance.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/provenance.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/provenance.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/provenance.md) at v10.1.0 – edit the repository, not this wiki.*
# Provenance
Reference-Commands-Raw-material-and-uploads.md
+10 -5
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/raw-material-and-uploads.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/raw-material-and-uploads.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/raw-material-and-uploads.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/raw-material-and-uploads.md) at v10.1.0 – edit the repository, not this wiki.*
# Raw material and uploads
@@ -72,6 +72,7 @@ tools/wikitool raw fetch --html incoming/post.html --url https://example.org/blo
- Fetch routes: the `fetch` section of the settings sends a host's pages another way. `session` fetches them with the cookies of the session the user stored with `raw login`, still with the built-in fetch and the bytes as received; the header adds `session: <host>` and never the session's content. The host's site handler names how a page shows it arrived signed in: no stored session, or a page without that sign (an expired session, a teaser), stops with exit 42 and writes nothing. Any other route names an external fetcher of this checkout: run without a shell with the URL as its last argument (120 s, 25 MiB), its stdout is the `.html` byte for byte, and the header says `fetched_by: wikitool raw fetch (fetcher: <name>)` and carries no `http_status`. A fetcher that fails is never replaced by the built-in fetch. A host no route names is fetched as before.
- Rendered pages: a site handler that declares `render` has its pages fetched through a headless browser - with the session, when the host has one - and the `.html` is the page as the browser drew it, not the bytes the server sent; the header says `rendered: true`. No page is rendered without that declaration, and an external fetcher's page never is. It needs the optional browser component; without it the fetch stops with exit 42. `--scope` fetches every page of the bundle the same way, and `_scope.json` names the `session`, `fetcher` or `rendered` it used.
- `--html` derives the `.md` beside a page saved to `incoming/` from a logged-in browser - the way past a paywall or a script-rendered page. Its header carries `derived` (when the text was derived) instead of `retrieved`, `final_url`, `http_status` and `content_type`, and the `.html` is left untouched.
- Length: a `CHECK` line names every `.md` this run wrote - and a `text/plain` or `text/markdown` response stored as received - that is longer than 50,000 characters, the threshold of `raw derive`: more than one ingest reads in one pass, so it is read in parts (`instructions/ingest-large-tree.md`).
- Success prints the `raw accept` line for the written files and the `source_url` for the source page.
**Properties**
@@ -194,11 +195,13 @@ tools/wikitool raw derive incoming/scans.zip
- A single file directly in `incoming/` becomes a folder named after its stem (or `--name`) holding `original.<ext>` - the original bytes, moved - and `derived.md`. The fixed names keep the path inside the path budget; the header of `derived.md` records the file's own name as `original_name`.
- A folder is derived in place: each file with a filter gets `<stem>.md` beside it, nothing is renamed, and the files without a filter are named in the output.
- `derived.md` starts with a fixed header (`derived_by`, `filter`, `engine`, `derived`, `derived_from`, `original_name` for a renamed file, `title`, then the fields of the format) followed by the derived text. The same bytes always give the same text; only `derived` differs between runs. `engine` names the tool version and, where a package reads the format, its version.
- Formats: web pages (`.html`), Word (`.docx`), PowerPoint (`.pptx`), Excel (`.xlsx`, `.xlsm`), e-mail (`.eml`) and ZIP archives (`.zip`), read with the standard library only; PDF (`.pdf`, `pypdf`) and Outlook messages (`.msg`, `olefile`), each read by a package in `tools/requirements.txt` that is imported only when such a file is derived. A missing package refuses that format alone, with the preflight hint; `wikitool doctor` shows one `import-filter-<name>` line per package-backed format. A PDF gets `<!-- page N -->` markers and a WARN when it yields almost no text (a scan; there is no OCR); an encrypted PDF is refused. A `.msg` is a container like an `.eml`; an embedded message inside it is named, not read. The text of a mail (`.eml` and `.msg` alike) is its plain text, unless that is under 10% of the length of the HTML text - a "plain text version not available" placeholder - in which case it is the HTML text and a WARN names both lengths. The other version of the text in a `multipart/alternative` is not an attachment.
- Formats: web pages (`.html`), Word (`.docx`), PowerPoint (`.pptx`), Excel (`.xlsx`, `.xlsm`), e-mail (`.eml`) and ZIP archives (`.zip`), read with the standard library only; PDF (`.pdf`, `pypdf` and `pdfminer.six`) and Outlook messages (`.msg`, `olefile`), each read by packages in `tools/requirements.txt` that are imported only when such a file is derived. A missing package refuses that format alone, with the preflight hint; `wikitool doctor` shows one `import-filter-<name>` line per package-backed format. A PDF gets `<!-- page N -->` markers and a WARN when it yields almost no text (a scan; there is no OCR); an encrypted PDF is refused. A `.msg` is a container like an `.eml`; an embedded message inside it is named, not read. The text of a mail (`.eml` and `.msg` alike) is its plain text, unless that is under 10% of the length of the HTML text - a "plain text version not available" placeholder - in which case it is the HTML text and a WARN names both lengths. The other version of the text in a `multipart/alternative` is not an attachment.
- A PDF also gets, where the document has them: an `outline` header field (one entry per outline item: title, level, page and the line of `derived.md` where that page's marker is), and markings in the text that follow the visible form only, in plain Markdown: `~~…~~` for strike-through, `**…**` for a highlighter stroke, bold type or a colour clearly unlike the running text (not under a link), `*…*` for italic. Bold, italic and colour count in running text only, never in program text, a table's grid or a shape the document repeats on its pages. A `markings` header field is the legend (at most three lines: form, places, pages; the `**` line names the highlighter colours); removing the markings gives the text the PDF has without them, and a tag never starts or ends on whitespace. A CHECK names the pages with a picture or drawing the text does not contain - not a background, a filler or a picture repeated across the document - and an `images` header field names the same pages, so that they outlast the output. An `unmatched` header field names the pages a WARN reports as left without markings because their text did not match the page layout. Both are written only when not empty, as page ranges (`6-8, 10`). A PDF without outline, markings, images and unmatched pages is derived as before.
- A workbook (`.xlsx`, `.xlsm`) gets its cells as files: beside its `.md` a folder `<md stem>.sheets/` (`derived.sheets/` for a single file) holds one `NN-<sheet>.tsv` per sheet - tab-separated per RFC 4180, UTF-8, row 1 of the file is row 1 of the sheet - and, for a sheet with formulas, `NN-<sheet>.formulas.tsv` listing cell, formula and last stored value. The `.md` is the overview: one section per sheet with its range, the file names and a preview. Formulas are not recalculated; a shared formula is written out per cell, and one that cannot be is named `shared:<master cell>` with a WARN. The folder counts against the path budget and is accepted with the rest; an existing one is never overwritten.
- A ZIP and a mail are containers. A ZIP at the top of `incoming/` becomes the folder `incoming/<stem>/` holding its files under their own names, each with a `<stem>.md` where a filter reads it; the ZIP is removed and the output names it with its sha256. A mail becomes a folder with `original.eml`, `derived.md` and its attachments under their own names; a ZIP attached to it is unpacked once into a folder of its own, and an attached mail is quoted in `derived.md`, not unpacked. A ZIP inside a ZIP, an absolute or `..` path, a symbolic link, an encrypted entry, more than 1000 entries, a file over 25 MiB or more than 250 MiB in total refuse the whole call. Hidden entries (`__MACOSX/`, `.DS_Store`, anything starting with a dot) are named, not written. A name taken twice, or reserved (`original.*` and `derived.md` at the top of a mail's folder, `_capture.json` anywhere), gets `-2`, `-3` before its extension, and the output lists each rename.
- A name already taken under `raw/` is a WARN, not a refusal: whether the folder is a later edition of that source or a separate one is the human's call. A later edition of a derived source is replaced file by file with `raw accept --replaces`; there is no `--replaces-bundle` for it.
- After a success the output prints the warnings the filters raised, at most three `CHECK` lines per format that was derived in this run - what to look at before the text is trusted - and the `raw accept` line for the result.
- Length: one more `CHECK` line names every `.md` this run wrote that is longer than 50,000 characters, counted in the file as written, header included - more than one ingest reads in one pass, so it is read in parts (`instructions/ingest-large-tree.md`). Characters, not tokens: they do not depend on a model's tokenizer. `raw fetch` and `raw pending` measure against the same value.
- Never edit `derived.md` by hand: a better derivation is a new `raw derive` of the original.
**Properties**
@@ -372,9 +375,10 @@ tools/wikitool raw pending --json
- Order: oldest first by modification time. A bundle or folder counts as new as its newest file; a tie goes by name. The mtime is when a document last changed only if it was copied with its timestamps kept (`cp -p`, `rsync -a`, an unpacked archive) - for a download or a `raw fetch` it is merely when it was dropped.
- Each candidate shows its path(s), kind, file count and mtime, and whether `raw accept` would take it as it stands - the same checks, minus `--fidelity`/`--authority`. One it would refuse is listed with the reason and skipped: it needs a human.
- The default is the first candidate `raw accept` would take, and the output names it.
- `--json` prints the same candidates in the same order: `kind`, `paths`, `files`, `mtime`, `acceptable`, `reason`, `default`, `hint`.
- `--json` prints the same candidates in the same order: `kind`, `paths`, `files`, `mtime`, `acceptable`, `reason`, `default`, `hint`, `length`.
- A lone file that a `raw derive` filter reads (a PDF, mail, Office file or web page) is listed with the `raw derive` line for it, in text output and as `hint` in JSON (`null` for every other candidate). The hint changes nothing about the order or the default.
- Reads directory listings and `lstat` only, never a file's content; an empty `incoming/` is exit 0 with nothing to do.
- A lone `.md` or `.txt` file longer than the length threshold of `raw derive` and `raw fetch` is listed with its length in characters, in text output and as `length` in JSON (`null` for every other candidate) - that text reaches a session with no derivation in front of it to print the length `CHECK`. Like the hint, it changes nothing about the order or the default.
- Reads directory listings and `lstat`, and a file's content only to measure a lone `.md`/`.txt` whose size in bytes is over the threshold - a UTF-8 text has no more characters than bytes; an empty `incoming/` is exit 0 with nothing to do.
**Properties**
@@ -529,6 +533,7 @@ tools/wikitool upload collect onedrive
- Everything a channel delivers lands in `quarantine/<id>/` unchanged, with a manifest that names `channel`, `handler` and `origin`; the submitter is `unknown`. Nothing reaches `incoming/` or `raw/` - that is `upload accept`, behind its gate, and an agent may run `collect` because collecting gives no trust.
- `folder`: only the top level of `path`. A file is taken once its modification time and, where the platform has it, its creation time are older than `settle_seconds`, its extension and size fit, and its name passes the filename rules. It is removed from the folder only after its copy is fully written and the source still has the size and sha256 that were read; if it changed meanwhile, nothing is kept and it stays. A file whose sha256 already waits in the quarantine is reported as a duplicate and removed from the folder, because its content is kept. Subdirectories, hidden files (names starting with `.` or `~$`, or the hidden or system attribute on Windows), symbolic links, other extensions and oversized files are left untouched and reported.
- `command`: the program of `commands.<name>` runs without a shell, with stdin closed, a 300 s timeout and an empty staging directory under `quarantine/.staging/` as its last argument. On exit 0 each regular file at the top of that directory with an allowed extension and size becomes a submission; on anything else nothing is taken and the end of stderr is printed. The staging directory is always removed.
- `folder` never empties backup or export data: a `path` that holds a `.chemenu-backup.json` (a machine's folder of the backup, any machine's) or a `.wikitool-export.json` (an export), is a bare git repository, or lies in one of these, is that channel's error - nothing is read and nothing removed. A `path` above such folders is fine, since only its top level is read.
- Channels are independent: when one fails the others still run, and the exit status is 1 if any channel or file failed. With no pulled channel it prints `nothing to collect` and exits 0.
- Exits 1, before it does anything, while an `mcp-upload/` of an earlier version still holds submissions, and when the `intake` section is malformed.
- Not idempotent: a second run finds the folder emptied.
@@ -545,7 +550,7 @@ tools/wikitool upload collect onedrive
- `0` - success
- `1` - Unknown channel name, an `mcp` channel as argument, a malformed `intake` section, or an `mcp-upload/` that still holds submissions
- `1` - A channel or a file failed (folder missing, program exited non-zero, timed out or could not be started, a source that could not be removed)
- `1` - A channel or a file failed (folder missing, a folder in backup or export data, program exited non-zero, timed out or could not be started, a source that could not be removed)
**See also**
Reference-Commands-Telemetry.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/telemetry.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/telemetry.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/telemetry.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/telemetry.md) at v10.1.0 – edit the repository, not this wiki.*
# Telemetry
Reference-Commands-Types-instructions-and-docs.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/types-instructions-and-docs.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/types-instructions-and-docs.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/types-instructions-and-docs.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/types-instructions-and-docs.md) at v10.1.0 – edit the repository, not this wiki.*
# Types, instructions and docs
Reference-Commands-Workshop-runs-and-session-budget.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/workshop-runs-and-session-budget.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/workshop-runs-and-session-budget.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/workshop-runs-and-session-budget.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/workshop-runs-and-session-budget.md) at v10.1.0 – edit the repository, not this wiki.*
# Workshop runs and session budget
Reference-Commands.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/commands/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/commands/README.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/commands/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/commands/README.md) at v10.1.0 – edit the repository, not this wiki.*
# Command reference
Reference-Configuration.md
+33 -15
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/configuration.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/configuration.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/configuration.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/configuration.md) at v10.1.0 – edit the repository, not this wiki.*
# Configuration
@@ -13,7 +13,7 @@ reads it with, so it cannot describe a field the tool does not take. What a sett
written by hand.
Three more files at the root are written by `wikitool` and never by hand: `.wikitool-release.json`,
`.wikitool-kb.json` and `.wikitool-tools.json` ([AGENTS.md invariant 1](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#invariants)).
`.wikitool-kb.json` and `.wikitool-tools.json` ([AGENTS.md invariant 1](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#invariants)).
The last one has its own section [below](#tool-paths), because the preflight that writes it is
something you run.
@@ -42,7 +42,7 @@ something you run.
| Variable | Purpose | Fallback |
|----------|---------|----------|
| `WIKI_AUTHOR` | Overrides the author name of new source pages | `git config user.name`. Without either, `new` stops with an `ERROR` |
| `WIKITOOL_SESSION_ID` | Scopes the Iteration Budget Gate to a task rather than a terminal, and names the directory a trace lands in | A session variable the harness sets itself, where one is known (for example `CLAUDE_CODE_SESSION_ID`), else the parent process id ([instructions/session-setup.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/session-setup.md)) |
| `WIKITOOL_SESSION_ID` | Scopes the Iteration Budget Gate to a task rather than a terminal, and names the directory a trace lands in | A session variable the harness sets itself, where one is known (for example `CLAUDE_CODE_SESSION_ID`), else the parent process id ([instructions/session-setup.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/session-setup.md)) |
| `WIKITOOL_UPDATE_URL` | The release feed `version check`, `version notes` and `dist upgrade --latest` ask | The value in `.wikitool-release.json`, else the origin's feed |
| `WIKITOOL_UPDATE_TOKEN` | A Gitea token for the release feed, sent only to downloads on the feed's own host | None. Not needed against the origin, only against a feed in a private repository |
| `WIKITOOL_TASKS_CONFIG` | Path to a tracker profile that `task`, `review` and `doctor` read instead of the `tasks`, `superproductivity` and `caldav` sections of the settings, for running one checkout against several trackers in turn ([below](#profiles)) | The sections in `wikitool.json` and `.wikitool-local.json`. A path to a file that does not exist is an error, never "no tracker configured" |
@@ -398,7 +398,7 @@ exits 42 for them.
<!-- wikitool:reference-config-intake -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `channels` | checkout, gate | object | yes | - | Channel name (lower-case letters, digits and hyphens) to channel. The name goes into the manifest of what the channel delivers and is the argument of `upload collect`. Every channel has `handler` (`mcp`, `folder` or `command`), `max_bytes` (the largest single file) and `allowed_extensions` (each starting with `.`, compared case-insensitively). Then per handler: `mcp` takes `identity_header` (the request header the authenticating middleware sets to the submitter's identity, default `X-Forwarded-User`) and, required, `quota_submissions_per_day` and `quota_bytes_per_day` (per submitter, over any rolling 24 hours); `folder` takes `path` (required: an absolute path outside the instance, read without descending into subdirectories) and `settle_seconds` (default 60: a file whose modification or creation time is younger is left alone); `command` takes `command` (required: the name of an entry of `commands`). At most one channel has the handler `mcp`. A field the handler does not take is an error. |
| `channels` | checkout, gate | object | yes | - | Channel name (lower-case letters, digits and hyphens) to channel. The name goes into the manifest of what the channel delivers and is the argument of `upload collect`. Every channel has `handler` (`mcp`, `folder` or `command`), `max_bytes` (the largest single file) and `allowed_extensions` (each starting with `.`, compared case-insensitively). Then per handler: `mcp` takes `identity_header` (the request header the authenticating middleware sets to the submitter's identity, default `X-Forwarded-User`) and, required, `quota_submissions_per_day` and `quota_bytes_per_day` (per submitter, over any rolling 24 hours); `folder` takes `path` (required: an absolute path outside the instance, read without descending into subdirectories; it must not be or lie in `<backup.target>/git` or `<backup.target>/<machine>`, and `upload collect` refuses a folder that holds or lies in any machine folder, bare repository or export) and `settle_seconds` (default 60: a file whose modification or creation time is younger is left alone); `command` takes `command` (required: the name of an entry of `commands`). At most one channel has the handler `mcp`. A field the handler does not take is an error. |
| `commands` | checkout, names a command | object | no | `{}` | Command name to command: an array of the program and its arguments, run by the `command` handler without a shell, with stdin closed and a timeout of 300 s, and a staging directory appended as the last argument. Exit 0 turns each regular file at the top level of that directory into a submission; anything else takes nothing over and prints the end of the program's stderr. |
- **Absent:** No channel: the `submit` tool is not registered and `upload collect` has nothing to collect. A valid state; `doctor` reports `OK`.
@@ -540,7 +540,7 @@ way, and is instance-tier, so every clone fetches the host the same way:
- `"session"` fetches the host's pages with the session the user stored with
`tools/wikitool raw login <host>`, through the same built-in fetch. The host needs a site handler
with a `session` block - its login page and how a page shows it arrived signed in
([sites/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/sites/CONTRACT.md)). No stored session, or a page without that sign,
([sites/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/sites/CONTRACT.md)). No stored session, or a page without that sign,
stops the fetch with exit 42 for the user to sign in again. The session is a secret: it is
stored in the secrets file [above](#tiered-settings) under `session_<host>` (every run of other
characters as one `_`, so `www.example.org` is `session_www_example_org`), or comes from
@@ -606,7 +606,7 @@ host name for one, do reach the synced folder.
<!-- wikitool:reference-config-backup -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `target` | checkout, gate | string | yes | - | The backup folder: an absolute path outside the instance. `backup init` creates `<target>/git` in it, `backup run` writes only to `<target>/<machine>/`. |
| `target` | checkout, gate | string | yes | - | The backup folder: an absolute path outside the instance. `backup init` creates `<target>/git` in it, `backup run` writes only to `<target>/<machine>/`. Everything else in the folder is the sync folder's: an `export` target or an `intake` folder may sit beside `git` and the machine folders (`<target>/Export`, `<target>/Inbox`), but none may be, or lie in, one of them. |
| `machine_data` | checkout, gate | boolean | no | `false` | Mirror this machine's files after every `publish` that pushed, and with `backup run`: `.wikitool-local.json`, `ENVIRONMENT.md`, what waits in `incoming/`, and `extra`. |
| `machine_name` | checkout | string | no | the host name | The name of this machine's folder under the target. Every character outside letters, digits, `-` and `_` becomes `-`, and leading and trailing `-` are dropped; a name that is empty afterwards, or `git` in any case, is an error. |
| `extra` | checkout, gate | array of strings | no | `[]` | Further files or directories to mirror, relative to the instance. One that resolves outside the instance, or into `.git/`, is an error. |
@@ -646,7 +646,7 @@ them into a staging directory and hands that to a converter of this checkout.
<!-- wikitool:reference-config-export -->
| Field | Tier | Type | Required | Default | Description |
|-------|------|------|----------|---------|-------------|
| `targets` | checkout, gate | object | yes | - | Target name (lower-case letters, digits and hyphens) to target; the name is the argument of `export run`. Every target has `handler` (`markdown`: the pages as Markdown files; `command`: the same Markdown run through a converter of this checkout), `path` (an absolute path outside the instance; it must not lie in or contain another target, an `intake` folder or `backup.target`) and `include` (an array of selections, each an array of `search --field` predicates: all predicates of a selection must hold, and a page that fits any selection is exported; there is no "everything"). Optional: `frontmatter` (`minimal`: only the export marker; `summary`, the default: also `title`, `type`, `summary` and `modified` where the page has them; `full`: the page's whole frontmatter), `links` (`relative`, the default: a wikilink to an exported page becomes a relative Markdown link; `text`: plain text; a link to a page that is not exported is always plain text) and `after_publish` (default false: refresh this target after every `publish` that committed). The handler `command` takes `command` (required: the name of an entry of `commands`). A field the handler does not take is an error. |
| `targets` | checkout, gate | object | yes | - | Target name (lower-case letters, digits and hyphens) to target; the name is the argument of `export run`. Every target has `handler` (`markdown`: the pages as Markdown files; `command`: the same Markdown run through a converter of this checkout), `path` (an absolute path outside the instance; it must not lie in or contain another target or an `intake` folder, and not be, lie in or contain `<backup.target>/git` or `<backup.target>/<machine>`, nor lie in any other machine's folder or a bare repository) and `include` (the string `"all"`: every page `search` knows, a page directly under `kb/` included and no generated or meta file; or an array of selections, each an array of `search --field` predicates: all predicates of a selection must hold, and a page that fits any selection is exported. A missing or empty `include` is an error, never "all"). Optional: `frontmatter` (`minimal`: only the export marker; `summary`, the default: also `title`, `type`, `summary` and `modified` where the page has them; `full`: the page's whole frontmatter), `links` (`relative`, the default: a wikilink to an exported page becomes a relative Markdown link; `text`: plain text; a link to a page that is not exported is always plain text), `index` (default false; true: also write `index.md` in the target's root and an `INDEX.md` per collection, and per area the catalog gives a file of its own, listing the exported pages only, with relative Markdown links whatever `links` says) and `after_publish` (default false: refresh this target after every `publish` that committed). The handler `command` takes `command` (required: the name of an entry of `commands`). A field the handler does not take is an error. |
| `commands` | checkout, names a command | object | no | `{}` | Command name to command: an array of the program and its arguments, run by the `command` handler without a shell, with stdin closed and a timeout of 300 s. The last two arguments appended are the directory holding the Markdown and an empty output directory. Exit 0 makes the regular files in the output directory the target's content; anything else leaves the target unchanged and prints the end of the program's stderr. |
- **Absent:** No export: `export run` stops with exit 1 naming the section, `publish` exports nothing and `doctor` reports `OK`.
@@ -674,6 +674,12 @@ Example:
"links": "relative",
"after_publish": true
},
"whole-kb": {
"handler": "markdown",
"path": "C:\\Users\\<user>\\OneDrive - <company>\\Chemenu-KB",
"include": "all",
"index": true
},
"as-word": {
"handler": "command",
"command": "pandoc-docx",
@@ -698,15 +704,27 @@ Example:
```
<!-- /wikitool:reference-config-export -->
Within one target, `include` is a list of selections, each a list of `search` predicates (`tags=docs`,
`entity_type=system`, `!source_url`): a page is exported when it satisfies every predicate of at least
one selection. Only what `include` selects is written - there is no way to exclude by default. The
syntax is checked when the section is read, the field names against the corpus when `export run`
runs. `path` must not lie in or contain the instance, another target, an `intake` folder or
`backup.target`.
Within one target, `include` is either the string `"all"` or a list of selections, each a list of
`search` predicates (`tags=docs`, `entity_type=system`, `!source_url`): a page is exported when it
satisfies every predicate of at least one selection. `"all"` selects every page `search` knows without
a predicate - one directly under `kb/` too, never a generated or meta file (`kb/index.md`, `kb/log.md`,
`kb/provenance.md`, `kb/CONTRACT.md`, `kb/CONVENTIONS.md`, an `INDEX.md`, a `COLLECTION.md`). Only what
`include` selects is written - there is no way to exclude by default, and a missing or empty `include`
is an error, never "all". The syntax is checked when the section is read, the field names against the
corpus when `export run` runs. `path` must not lie in or contain the instance, another target, an
`intake` folder or `backup.target`.
With `"index": true` a run also writes `index.md` in the root of the target and an `INDEX.md` per
collection - and per area the catalog under `kb/` would give a file of its own. They list the exported
pages only (Page, Type, Summary, Last Modified), link with relative Markdown links whatever `links`
says, and carry no date of the day: Last Modified is the page's own `modified`, `date` or `created`,
empty where it has none. The column and heading labels are English. Each starts with a frontmatter
block whose only key is `wikitool_export` (`kind=index`), so `raw accept` refuses it like a page. A
page `<collection>/index.md` beside the generated `<collection>/INDEX.md` stops the target, since a
synced folder on Windows would make them one file.
Every Markdown file starts with a frontmatter block whose first key is `wikitool_export`, carrying
the kind, the instance and the commit of the newest selected page. `frontmatter` decides what follows:
the kind, the instance and the newest commit that touched that page. `frontmatter` decides what follows:
`minimal` nothing, `summary` the title, type, summary and modified date, `full` the page's whole
frontmatter. Wikilinks between exported pages become relative Markdown links (`links: relative`) or
plain text (`links: text`); a link to a page that is not exported is always plain text. Footnote
@@ -733,4 +751,4 @@ more. A command that needs the setting such a file held stops with exit 42 and n
ignores it. `config migrate` moves the content into the sections above (the upload file becomes the channel `mcp` of `intake`), each value into the file its
tier names and each secret into the secrets file, and then removes the old files; it is run once per
checkout. `.wikitool-tools.json` is not among them and stays. The migration step for an instance is
[10.0.0-settings-tiers.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/instructions/migrations/10.0.0-settings-tiers.md).
[10.0.0-settings-tiers.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/instructions/migrations/10.0.0-settings-tiers.md).
Reference-Exit-codes.md
+2 -2
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/exit-codes.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/exit-codes.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/exit-codes.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/exit-codes.md) at v10.1.0 – edit the repository, not this wiki.*
# Exit codes and gates
@@ -6,7 +6,7 @@ Which `wikitool` commands can stop and wait for a person - exit 42 - and for whi
from the command records.
<!-- wikitool:reference-exit-codes -->
What each exit code means, and how a caller reacts to it, is [AGENTS.md § Tool error contract](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#tool-error-contract). What a gate is and why it is enforced in code is [AGENTS.md § Gates](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/AGENTS.md#gates). This page lists which commands declare a gate, and every cause for which a command exits 42.
What each exit code means, and how a caller reacts to it, is [AGENTS.md § Tool error contract](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#tool-error-contract). What a gate is and why it is enforced in code is [AGENTS.md § Gates](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/AGENTS.md#gates). This page lists which commands declare a gate, and every cause for which a command exits 42.
## Gates by command
Reference-Page-types.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/page-types.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/page-types.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/page-types.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/page-types.md) at v10.1.0 – edit the repository, not this wiki.*
# Page types
Reference-Skills.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/skills.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/skills.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/skills.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/skills.md) at v10.1.0 – edit the repository, not this wiki.*
# Skill catalogue
Reference.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/reference/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/reference/README.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/reference/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/reference/README.md) at v10.1.0 – edit the repository, not this wiki.*
# Reference
Tutorials-First-source-to-first-answer.md
+2 -2
@@ -1,4 +1,4 @@
*Generated from [`docs/tutorials/first-source-to-first-answer.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/tutorials/first-source-to-first-answer.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/tutorials/first-source-to-first-answer.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/tutorials/first-source-to-first-answer.md) at v10.1.0 – edit the repository, not this wiki.*
# From your first source to your first answer
@@ -86,7 +86,7 @@ verbatim, reporting
```
The agent never guesses these two. Only you know them, and only now: once the file is in the wiki,
nobody can tell any more where it came from. [raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/raw/CONTRACT.md#capture-fields-fidelity-and-authority)
nobody can tell any more where it came from. [raw/CONTRACT.md](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/raw/CONTRACT.md#capture-fields-fidelity-and-authority)
lists every value, for the sources you add later.
## 5. Let the agent publish
Tutorials-Sample-source.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/tutorials/sample-source.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/tutorials/sample-source.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/tutorials/sample-source.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/tutorials/sample-source.md) at v10.1.0 – edit the repository, not this wiki.*
# Home lab backups - how they work now
Tutorials.md
+1 -1
@@ -1,4 +1,4 @@
*Generated from [`docs/tutorials/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.0.0/docs/tutorials/README.md) at v10.0.0 – edit the repository, not this wiki.*
*Generated from [`docs/tutorials/README.md`](https://gitea.nehmer.net/torben/chemenu/src/tag/v10.1.0/docs/tutorials/README.md) at v10.1.0 – edit the repository, not this wiki.*
# Tutorials