feat: docs/ carries the operator's how-to guides, a configuration reference and the architecture and telemetry explanations carved out of README.md, INSTALL.md, INSTALL-MCP.md and EVALS.md; the last two retired (#190)
Files changed: - .gitea/workflows/ci.yml - .gitignore - .vibe/hooks.toml - AGENTS.md - CHANGES.md - EVALS.md - INSTALL-MCP.md - INSTALL.md - README.md - VERSION - docs/dev/development.md - docs/explanation/README.md - docs/explanation/architecture.md - docs/explanation/language-boundaries.md - docs/explanation/telemetry-and-evaluation.md - docs/how-to/README.md - docs/how-to/browse-with-obsidian.md - docs/how-to/capture-repository-docs.md - docs/how-to/export-guidelines.md - docs/how-to/ingest-a-source.md - docs/how-to/maintain-the-wiki.md - docs/how-to/query-the-wiki.md - docs/how-to/run-the-weekly-review.md - docs/how-to/set-up-the-mcp-server.md - docs/how-to/trace-and-score-sessions.md - docs/how-to/troubleshoot.md - docs/how-to/update-an-instance.md - docs/reference/README.md - docs/reference/commands/distribution-and-versioning.md - docs/reference/commands/instance-health.md - docs/reference/commands/raw-material-and-uploads.md - docs/reference/commands/telemetry.md - docs/reference/commands/types-instructions-and-docs.md - docs/reference/configuration.md - instructions/dev/dev-setup.md - instructions/dev/doc-pull-through.md - instructions/dev/stack-build/SKILL.md - instructions/dev/testing-conventions.md - instructions/dev/version-parts.md - instructions/setup-instance.md - instructions/upgrade-instance.md - reports/CONTRACT.md - tools/.coveragerc - tools/CONTRACT.md - tools/README.md - tools/chemenu/commands/dist_cmd.py - tools/chemenu/commands/docs_verify.py - tools/chemenu/commands/doctor.py - tools/chemenu/commands/eval_cmd.py - tools/chemenu/commands/git_publish.py - tools/chemenu/commands/upload_cmd.py - tools/chemenu/commands/version_cmd.py - tools/chemenu/evals/__init__.py - tools/chemenu/evals/scorecard.py - tools/chemenu/evals/trajectory.py - tools/chemenu/telemetry/policy.py - tools/chemenu/tests/test_dist_cmd.py - tools/chemenu/tests/test_git_publish.py - tools/chemenu/tests/test_instructions_shell.py - tools/chemenu/tests/test_pipeline_l0.py - tools/chemenu/toc.py Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
torbenandClaude Opus 5.5 committed
1 parent
cff7df4ffa
commit
f9056c04f7
61 files changed
+2169
-1772
No files matched your search
+51
-1
@@ -20,6 +20,7 @@ them. The reason is a comment at `dist_cmd.DOCS_DEV_ONLY`, so that a later sessi
|
||||
- [A release feed in a non-public repository](#a-release-feed-in-a-non-public-repository)
|
||||
- [The release workflow](#the-release-workflow)
|
||||
- [The CI half](#the-ci-half)
|
||||
- [How much of the stack the suite reaches](#how-much-of-the-stack-the-suite-reaches)
|
||||
- [Stack development as a session type of its own](#stack-development-as-a-session-type-of-its-own)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
@@ -47,7 +48,8 @@ What a clone has and an instance does not:
|
||||
- **A demo persona** in `USER.md`/`SOUL.md`. `doctor` reports both as filled in. They describe
|
||||
the demo setup, not you.
|
||||
- **Telemetry on.** Without `.wikitool-release.json` the clone is the measuring station the stack
|
||||
evaluates itself with ([EVALS.md](../../EVALS.md) § "Whether it runs at all").
|
||||
evaluates itself with
|
||||
([telemetry-and-evaluation.md](../explanation/telemetry-and-evaluation.md#whether-it-runs-at-all)).
|
||||
- **`instructions/dev/`, `docs/dev/`, `commonplace/` and `.gitea/`.** `dist export` ships none
|
||||
of them.
|
||||
|
||||
@@ -180,6 +182,54 @@ repository by hand. What the suite writes, how to run it against a tracker of yo
|
||||
a red run means is in
|
||||
[instructions/dev/tracker-testing.md](../../instructions/dev/tracker-testing.md).
|
||||
|
||||
### How much of the stack the suite reaches
|
||||
|
||||
Coverage is measured in CI - `pytest --cov`, config in `tools/.coveragerc`, HTML and XML
|
||||
uploaded as the `coverage-<run id>` artifact of every run.
|
||||
**Fetch that artifact from the run's own page, not from the API**: `upload-artifact@v3` writes
|
||||
through the older artifact API, and the Actions artifact REST endpoints answer `total_count: 0`
|
||||
for a run whose artifact the run page offers for download. The upload works; only the listing
|
||||
does not see it. Do not re-derive this, and do not read the empty list as a failed upload.
|
||||
|
||||
It is enforced at a floor of **85%** (`fail_under` in `tools/.coveragerc`), which is what a red
|
||||
suite from this axis means: coverage actually fell, not that a wrapper was added. The floor was
|
||||
set only after the number had been watched - it was deliberately held back for exactly that, and
|
||||
the two points between 85 and the measured 87.0% are the room the taxonomy below asks for. A
|
||||
threshold at the measured number goes red on the next thin Typer wrapper, and a threshold that
|
||||
goes red for a non-reason gets lowered rather than earned.
|
||||
|
||||
**Measured 2026-09-04, stack 4.7.1: 87.0% of 6498 statements across `chemenu/`, 975 tests** -
|
||||
CI run 163. The first measurement, at stack 1.8.1 on 2026-08-31, was 86.9% of 5105 statements
|
||||
over 730 tests (CI run 87). Both are what CI reported, never a local run: the local number
|
||||
preceding a release measures a tree that is one commit short of the published one.
|
||||
|
||||
The pair says more than either number does. Between them the measured code grew by a quarter
|
||||
and the suite by a third, and the quota moved by a tenth of a point - which is the observation a
|
||||
threshold was waiting for, rather than the total itself. Reproduce either with
|
||||
`cd tools && .venv/bin/python -m pytest -q --cov` (needs `pytest-cov`, which is CI-only and
|
||||
deliberately absent from `tools/requirements.txt` - an instance runs the wiki, it does not
|
||||
measure this suite).
|
||||
|
||||
The total stays the least interesting number here. What the report is for is *which* modules sit
|
||||
low, and three kinds have to be told apart before any of it turns into work:
|
||||
|
||||
- **Thin Typer wrappers**, where the logic lives beside them and is tested there:
|
||||
`eval_cmd.py` (36%), `types_cmd.py` (40%), `search.py` (49%), `cli.py` (54%),
|
||||
`links_cmd.py` (61%). Low coverage on a wrapper is evidence of a good cut, not of a missing
|
||||
test - `search.py`'s uncovered block is its command body alone, while the backends under
|
||||
`chemenu/search/` that do the work sit between 91% and 98%.
|
||||
- **Code that reaches the network or the filesystem's outside**, where the interesting half is
|
||||
already injectable and tested through the seam: `version.py`'s `fetch_latest()` takes a
|
||||
`fetcher` parameter for exactly that. The real `urllib` lines are exercised too, but only
|
||||
against a local `http.server` in `test_dist_upgrade.py` (`dist upgrade --latest`) - never
|
||||
against a real feed, which stays uncovered on purpose.
|
||||
- **Genuine gaps**, where uncovered lines are logic nobody exercises: `provenance_cmd.py`
|
||||
(44%), `migrate_cmd.py` (65%), `type_resolver.py` (79%). This is the list worth reading, and
|
||||
the only one of the three that has not moved while everything around it did:
|
||||
`provenance_cmd.py` sits where it sat, and `migrate_cmd.py` fell from 71% because it grew and
|
||||
its new lines arrived untested. The floor freezes this; it does not close it. Closing it is
|
||||
Gitea #51.
|
||||
|
||||
## Stack development as a session type of its own
|
||||
|
||||
Three skills (`instructions/dev/`, present only in this origin repository) carry a session that
|
||||
|
||||
@@ -4,6 +4,10 @@ Why the stack is built the way it is. These pages are background for humans and
|
||||
read by link or on request, never as instruction, and they carry no normative sentence - a rule
|
||||
lives in `AGENTS.md` or a `CONTRACT.md`, and a page here only gives its reasons.
|
||||
|
||||
- [architecture.md](architecture.md) - what the stack is made of, and how its parts divide the
|
||||
work
|
||||
- [telemetry-and-evaluation.md](telemetry-and-evaluation.md) - how the stack observes what an
|
||||
agent did, and how that record turns into a score
|
||||
- [pipeline-rationale.md](pipeline-rationale.md) - why the pipeline has four stages
|
||||
- [ownership-and-templates.md](ownership-and-templates.md) - why a `.template` split exists, and
|
||||
why an instance comes only from a release
|
||||
|
||||
@@ -0,0 +1,279 @@
|
||||
# Architecture
|
||||
|
||||
What Chemenu is made of, and how the parts divide the work. For why the pipeline has exactly four
|
||||
stages, see [pipeline-rationale.md](pipeline-rationale.md). This page is the map those reasons
|
||||
apply to.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [A wiki that is compiled, not searched](#a-wiki-that-is-compiled-not-searched)
|
||||
- [The language of the pages](#the-language-of-the-pages)
|
||||
- [The tree](#the-tree)
|
||||
- [Collections and areas](#collections-and-areas)
|
||||
- [Contracts and READMEs](#contracts-and-readmes)
|
||||
- [Who does what](#who-does-what)
|
||||
- [The compiler: `tools/wikitool`](#the-compiler-toolswikitool)
|
||||
- [The MCP read server](#the-mcp-read-server)
|
||||
- [Built for IT work](#built-for-it-work)
|
||||
- [Telemetry and evaluation](#telemetry-and-evaluation)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## A wiki that is compiled, not searched
|
||||
|
||||
Chemenu is a pattern for a personal knowledge base maintained by an LLM. A plain retrieval setup goes
|
||||
back to the raw documents for every question. Here, the LLM **builds and maintains a persistent
|
||||
wiki** incrementally instead, and the wiki compounds over time. Knowledge is compiled once and kept
|
||||
current, not re-derived for every question.
|
||||
|
||||
That one decision shapes everything else. Raw material is kept unchanged as evidence, and compiled
|
||||
pages cite it. Every mechanical step of the compilation - frontmatter, cross-references, the catalog,
|
||||
the log, publishing - is done by a deterministic tool, never by hand, so it comes out the same way
|
||||
every time.
|
||||
|
||||
## The language of the pages
|
||||
|
||||
Source material in `raw/` is never touched and stays in whatever language it arrived in. The
|
||||
compiled pages under `kb/` are written in the instance's own language. The origin repository's demo
|
||||
corpus, for instance, is German. What stays in its original form inside a page is everything that
|
||||
is an *identifier* rather than prose: page titles, section headings the tool owns, wikilink targets,
|
||||
citation ids, schema enum values, tags, commands, paths and code. So `GitOps Ownership Model` and a
|
||||
German section heading sit in the same page without contradiction.
|
||||
|
||||
Which lines count as identifiers is the stack's rule,
|
||||
[kb/CONTRACT.md § Language and identifiers](../../kb/CONTRACT.md#language-and-identifiers).
|
||||
*Which language* the prose is in, and what the tool-owned headings are called, is the instance's own
|
||||
`kb/CONVENTIONS.md`. That is a per-instance decision, not a property of the pattern, so it lives in a
|
||||
file the instance owns rather than in one the stack ships. A new instance picks its language when it
|
||||
fills in `kb/CONVENTIONS.md`, before the first ingest. Why the control plane stays English
|
||||
regardless: [language-boundaries.md](language-boundaries.md).
|
||||
|
||||
## The tree
|
||||
|
||||
```
|
||||
chemenu/
|
||||
├── AGENTS.md # Control plane: invariants, file naming, routing, gates
|
||||
├── CLAUDE.md # Claude Code only: imports AGENTS.md, links the one Claude-Code-only decision (model/effort). No rules of its own
|
||||
├── README.md # Landing page for humans
|
||||
├── INSTALL.md # Human-readable install: one release, one sentence to the agent
|
||||
├── CHANGES.md # Changelog for the stack itself
|
||||
├── USER.md # Who operates this instance - context, never instructions
|
||||
├── SOUL.md # How this instance sounds. AGENTS.md always wins over it
|
||||
├── ENVIRONMENT.md # Optional, gitignored: this checkout's harness, MCP servers, remotes
|
||||
├── *.md.template # Unfilled USER/SOUL/ENVIRONMENT - what a distribution ships instead
|
||||
├── .gitignore # Anchored so nothing under raw/, kb/ or work/ is ever excluded;
|
||||
│ # incoming/ is the one directory excluded the other way round
|
||||
├── .github/hooks/ # Copilot CLI hooks - session tracing
|
||||
├── .vibe/ # Mistral Vibe hooks + the repo's telemetry policy
|
||||
├── docs/ # Human documentation
|
||||
│ ├── tutorials/ # learning: a guided first walk
|
||||
│ ├── how-to/ # doing: one task per page
|
||||
│ ├── reference/ # looking up: commands, exit codes, skills, page types, configuration
|
||||
│ └── explanation/ # understanding: why it is built this way (this page)
|
||||
├── instructions/ # CONTROL: everything an agent is told to do
|
||||
│ ├── CONTRACT.md # Instruction vs. skill, publishing, writing standard
|
||||
│ ├── bootstrap.md # Prepare a further checkout of an instance
|
||||
│ ├── gates.md # What to do when a gate refuses a call
|
||||
│ ├── german-terminology.md # Which words stay English in German prose; register
|
||||
│ ├── session-setup.md
|
||||
│ ├── page-lifecycle.md
|
||||
│ ├── publish-cycle.md
|
||||
│ ├── ingest-large-tree.md
|
||||
│ ├── ingest-queue.md # Reviewing a submission before wikitool upload accept promotes it
|
||||
│ └── wiki-*/SKILL.md # Skills - copied into .agents/skills/ and .claude/skills/
|
||||
├── mcp-upload/ # QUARANTINE (optional): the MCP `submit` tool's write path, gitignored -
|
||||
│ # read by no command in the ordinary pipeline; a human reviews it with
|
||||
│ # `wikitool upload list/show/accept/reject`
|
||||
├── incoming/ # INBOX: content gitignored - drop a file or one folder per source here, `raw accept` promotes it
|
||||
├── raw/ # INPUT: immutable, untrusted source material
|
||||
│ ├── CONTRACT.md # Date shard, capture fields, immutability, untrusted content
|
||||
│ ├── 2026/09/ # Where `raw accept` puts a file: the month it was accepted
|
||||
│ ├── articles/ # The old type directories: still valid paths, never moved,
|
||||
│ ├── documents/ # but nothing new is ever routed into them again
|
||||
│ ├── notes/
|
||||
│ └── assets/
|
||||
├── types/ # SCHEMA: the global type surface. Not a collection
|
||||
│ ├── type-spec.md # Root contract: anatomy, placement, adding a type
|
||||
│ ├── type-guidance.md # Contract for the *.guidance.md files below
|
||||
│ ├── entity.md # Entity type config + template (+ .schema.yaml)
|
||||
│ ├── entity.guidance.md # Its stack-owned authoring prose, shipped verbatim
|
||||
│ ├── entity.person.md # Subtype template: what `new` scaffolds for entity_type=person
|
||||
│ ├── concept.md # Concept type config + template (+ .guidance.md, + concept.decision.md)
|
||||
│ ├── source.md # Source type config + template (+ .guidance.md)
|
||||
│ ├── comparison.md # Comparison type config + template (+ .guidance.md)
|
||||
│ ├── project.md # Project (Vorhaben) type config + template, no guidance file
|
||||
│ ├── instruction.md # Instruction type - lives outside kb/ via `root: repo`
|
||||
│ └── lint-report.md # Contract-only: describes reports/, owns no directory
|
||||
├── kb/ # OUTPUT: compiled knowledge. A namespace, not a collection
|
||||
│ ├── CONTRACT.md # Collections, naming, tone, linking, provenance
|
||||
│ ├── index.md # Generated catalog *map*: counts and pointers
|
||||
│ ├── log.md # Generated chronological audit log
|
||||
│ ├── provenance.md # Generated raw-file reverse index
|
||||
│ ├── entities/ # COLLECTION.md + INDEX.md + areas below
|
||||
│ │ ├── codebases/
|
||||
│ │ ├── systems/
|
||||
│ │ ├── tools/ # own INDEX.md once past 50 pages
|
||||
│ │ ├── technologies/
|
||||
│ │ ├── people/
|
||||
│ │ └── organizations/
|
||||
│ ├── concepts/ # COLLECTION.md + INDEX.md + areas below
|
||||
│ │ ├── architectures/
|
||||
│ │ ├── patterns/
|
||||
│ │ ├── protocols/
|
||||
│ │ ├── workflows/
|
||||
│ │ ├── decisions/
|
||||
│ │ └── problems/
|
||||
│ ├── sources/ # COLLECTION.md + INDEX.md + areas below
|
||||
│ │ ├── transcripts/
|
||||
│ │ ├── analyses/
|
||||
│ │ ├── articles/
|
||||
│ │ ├── documents/
|
||||
│ │ ├── notes/
|
||||
│ │ ├── trackers/
|
||||
│ │ └── unclassified/
|
||||
│ ├── comparisons/ # COLLECTION.md - comparison pages, no subtype axis
|
||||
│ └── gtd/ # COLLECTION.md + INDEX.md + areas below
|
||||
│ ├── haus/
|
||||
│ ├── finanzen/
|
||||
│ └── technik/
|
||||
├── work/ # WORKSHOP: one directory per multi-session run, tracked
|
||||
│ └── CONTRACT.md # Run keys, required files, how a run closes
|
||||
├── reports/ # DERIVED: lint reports, traces, eval scores. Gitignored
|
||||
│ └── CONTRACT.md
|
||||
└── tools/ # COMPILER: the wikitool CLI
|
||||
├── CONTRACT.md # Command records (generated) and index, maintenance schedule
|
||||
└── README.md # How wikitool is built and how to change it
|
||||
```
|
||||
|
||||
The collections and areas under `kb/` shown here are the origin repository's. An instance's own come
|
||||
from its type-specs, as described below.
|
||||
|
||||
<!-- dist:strip-start -->
|
||||
Only in the origin repository (`tools/CONTRACT.md` explains how an instance comes to lack them):
|
||||
|
||||
```
|
||||
├── docs/dev/ # Human-readable: development environment and release workflow
|
||||
└── commonplace/ # Vendored, read-only knowledge base
|
||||
```
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
## Collections and areas
|
||||
|
||||
A directory under `kb/` is a **collection** exactly when it holds a `COLLECTION.md`. A subdirectory
|
||||
inside one is an **area**, which inherits it, and that is as deep as a page goes. Nothing nests
|
||||
below an area, because the generated catalog reads exactly two path segments under `kb/`, and it
|
||||
would silently fold a deeper page into the area. `kb/CONTRACT.md` § Collections has the rule, and
|
||||
`wikitool lint` reports a violation as a hard error.
|
||||
|
||||
Which areas a collection has is not chosen page by page. A type-spec's `layout:` maps its subtype
|
||||
field onto directories, and `wikitool new` writes each page straight into the one its subtype names.
|
||||
That is also what gives the catalog's shard threshold any effect: `index rebuild` splits **per
|
||||
area**, so a collection with no areas keeps one table however large it grows. Once such a collection
|
||||
is past the threshold, `wikitool lint` reports it as a recommendation rather than an error, together
|
||||
with the split its subtype field would produce. It stays quiet when the split would not actually
|
||||
help.
|
||||
|
||||
`kb/comparisons/` is the worked example of a collection that stays flat: it has no subtype field for
|
||||
a `layout:` to key on at all. A *lopsided* subtype field is a different case, and it gets fixed
|
||||
rather than left flat. `kb/sources/` looked lopsided only because `source_type` had a schema
|
||||
`default:`, which the compiler applied whenever nobody chose a value. Once the default was removed
|
||||
and the pages reclassified, it split into six real areas.
|
||||
|
||||
Every page type the stack ships, with its subtypes, its directories and its required fields, is in
|
||||
[page-types.md](../reference/page-types.md). An instance adopts those type-specs and may change
|
||||
them; `tools/wikitool types list` and `tools/wikitool types describe <type>` show what one instance
|
||||
has.
|
||||
|
||||
## Contracts and READMEs
|
||||
|
||||
`COLLECTION.md` never appears outside `kb/`. The other layers carry a `CONTRACT.md` or a root
|
||||
type-spec instead. A stage may carry both a `README.md` and a `CONTRACT.md`, because they have
|
||||
different readers: the README is for humans working *on* that layer, and the contract binds an agent
|
||||
working *with* it. `AGENTS.md` § File naming says which file is for which reader, and
|
||||
`wikitool docs verify` checks it.
|
||||
|
||||
## Who does what
|
||||
|
||||
**You** curate sources by dropping them into `incoming/`, ask questions, review what changed in
|
||||
`kb/log.md` and `kb/index.md`, and steer the agent on what to emphasise or look into. The
|
||||
[how-to guides](../how-to/README.md) cover each of these.
|
||||
|
||||
**The agent** follows `AGENTS.md`, the cross-cutting schema and policy, and runs the step-by-step
|
||||
procedures as skills. They are published into `.agents/skills/`, and mirrored into
|
||||
`.claude/skills/` for Claude Code, by `tools/wikitool instructions sync`. Which skills ship, what
|
||||
each does and which requests start it: [skills.md](../reference/skills.md).
|
||||
|
||||
**`tools/wikitool`** does every mechanical step a skill needs: scaffolding, renaming and deleting
|
||||
pages, cross-references, regenerating the catalog, log and provenance index, structural linting,
|
||||
decay math, and publishing. The agent calls it and never does that work by hand. Along the way it
|
||||
maintains:
|
||||
|
||||
- `kb/index.md`, the catalog, always current;
|
||||
- `kb/log.md`, the complete audit trail;
|
||||
- `kb/provenance.md`, the reverse index from raw files to pages;
|
||||
- pages of every type, each in the directory its type-spec names;
|
||||
- lint reports, session traces and eval scores in `reports/`, which is gitignored.
|
||||
|
||||
## The compiler: `tools/wikitool`
|
||||
|
||||
Every command carries one data record kept next to its code: synopsis, properties, copyable
|
||||
examples, each exit cause with what to do about it, prohibitions, and notes on its behaviour. The
|
||||
same record is rendered four ways:
|
||||
|
||||
- `tools/wikitool <command> -h` prints it, and `tools/wikitool -h` prints a one-line index of every
|
||||
command;
|
||||
- a command that fails with exit 1 prints the record's reactions on stderr, right under its `ERROR`
|
||||
line;
|
||||
- [tools/CONTRACT.md](../../tools/CONTRACT.md) holds a generated copy for agents, together with the
|
||||
maintenance schedule;
|
||||
- [the command reference](../reference/commands/README.md) is a generated copy for humans.
|
||||
|
||||
`tools/wikitool docs verify` checks the copies, and every command's flags, against the CLI in both
|
||||
directions. [tools/README.md](../../tools/README.md) is the other half: how the CLI is built and how
|
||||
to add a command. `AGENTS.md` holds the invariants that say when each command is mandatory.
|
||||
|
||||
The wiki is also a git repository, and `tools/wikitool publish` is the one way its changes are
|
||||
committed and pushed. That gives every change a version history, and the gates in front of
|
||||
`publish` decide what needs a human to look first
|
||||
([why-gates-are-code.md](why-gates-are-code.md)).
|
||||
|
||||
<!-- dist:strip-start -->
|
||||
Extending `tools/wikitool`, the type schema or the instruction layer itself is a separate kind of
|
||||
session, with rules of its own: the `stack-dev`, `stack-build` and `stack-close` skills under
|
||||
`instructions/dev/`, which only the origin repository has.
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
## The MCP read server
|
||||
|
||||
The terminal is not the only way in. `tools/chemenu/mcp/` serves the same wiki read-only over MCP -
|
||||
`search`, `types`, `describe_type`, `lint` and `status` - so a consumer that is not a shell on this
|
||||
machine can ask the same questions and get the same answers. Literally the same: the CLI and the
|
||||
server are two adapters over one core, and a golden test holds their output together rather than
|
||||
trusting that it agrees.
|
||||
|
||||
There is no tool that writes to `kb/`, and not because one is filtered out of a list. The server
|
||||
imports nothing under `chemenu/commands/`, so `new`, `publish` and the rest cannot be reached from
|
||||
it. Every answer carries the commit it was computed from, so a checkout that has fallen behind gives
|
||||
a visibly stale answer instead of a confident wrong one. The server's dependency is deliberately
|
||||
not in `requirements.txt`: an instance that only uses the CLI should not have to install a web stack
|
||||
for it.
|
||||
|
||||
Setting it up: [set-up-the-mcp-server.md](../how-to/set-up-the-mcp-server.md).
|
||||
|
||||
## Built for IT work
|
||||
|
||||
The shipped defaults are tailored to IT work:
|
||||
|
||||
- **entity types** for software development and systems - codebases, systems, tools, technologies;
|
||||
- **relationship labels** such as "depends on", "uses" and "implements". The vocabulary is in the
|
||||
instance's `kb/CONVENTIONS.md`, because it belongs to the instance rather than the stack;
|
||||
- **templates** per page type, and per subtype where its pages need a shape of their own, such as a
|
||||
person or a decision record (ADR);
|
||||
- **guidelines** for documenting technical decisions, which can be exported back into the
|
||||
repositories they are about ([export-guidelines.md](../how-to/export-guidelines.md));
|
||||
- **cross-reference patterns** for code and architecture.
|
||||
|
||||
## Telemetry and evaluation
|
||||
|
||||
How the stack observes what an agent did, and how that record becomes a score, is its own page:
|
||||
[telemetry-and-evaluation.md](telemetry-and-evaluation.md).
|
||||
@@ -126,7 +126,7 @@ it did.
|
||||
|
||||
## The stack's human documentation
|
||||
|
||||
`README.md`, `INSTALL*.md`, `EVALS.md` and everything under `docs/` are read by humans, and the
|
||||
`README.md`, `INSTALL.md` and everything under `docs/` are read by humans, and the
|
||||
reader axis alone would put them in the operator's language. They are English anyway, and the
|
||||
reason is not the reader but the fact that there is no single one. This documentation belongs to
|
||||
the stack, not to an instance: it ships identically to every instance, whatever each one's
|
||||
|
||||
@@ -0,0 +1,354 @@
|
||||
# Telemetry and evaluation
|
||||
|
||||
How the stack observes what an agent did, and how that record turns into a score. This page explains
|
||||
the design and points at the code. The rules an agent follows live in
|
||||
[tools/CONTRACT.md](../../tools/CONTRACT.md) and [reports/CONTRACT.md](../../reports/CONTRACT.md).
|
||||
Turning tracing on and reading a session back is a how-to:
|
||||
[trace-and-score-sessions.md](../how-to/trace-and-score-sessions.md).
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Why, beyond the unit tests](#why-beyond-the-unit-tests)
|
||||
- [Architecture](#architecture)
|
||||
- [The trace](#the-trace)
|
||||
- [Whether it runs at all](#whether-it-runs-at-all)
|
||||
- [What never reaches a trace](#what-never-reaches-a-trace)
|
||||
- [Evaluation levels](#evaluation-levels)
|
||||
- [L0 lives in the test suite, not in a separate harness](#l0-lives-in-the-test-suite-not-in-a-separate-harness)
|
||||
- [Scoring a session](#scoring-a-session)
|
||||
- [A worked example, and where the check moved to](#a-worked-example-and-where-the-check-moved-to)
|
||||
- [Status](#status)
|
||||
- [The agent runner, and why it is not here yet](#the-agent-runner-and-why-it-is-not-here-yet)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Why, beyond the unit tests
|
||||
|
||||
The test suite that ships with the origin repository checks the **compiler**: given this input, does
|
||||
`wikitool` produce that output? It says nothing about the two things that actually go wrong in
|
||||
practice: whether the *agent* followed the contracts, and whether the pages it wrote are any good.
|
||||
|
||||
Those need a different kind of check. Two notes in the
|
||||
[Commonplace](https://github.com/zby/commonplace) research base, which the origin repository
|
||||
vendors, supply the theory:
|
||||
|
||||
- *oracle-strength-spectrum* - correctness checks form a gradient, from hard (deterministic) to none
|
||||
(vibes). The engineering move is to *harden* oracles step by step, not to reach straight for a
|
||||
judge.
|
||||
- *evaluation-automation-is-phase-gated-by-comprehension* - comprehension, then specification,
|
||||
then generalisation. A judge built before anyone has read real failures optimises a proxy.
|
||||
|
||||
That ordering is why this page describes a lot of telemetry and only a little scoring. Reading real
|
||||
traces is the first phase, and it cannot be skipped.
|
||||
|
||||
## Architecture
|
||||
|
||||
Three sources, three different jobs.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph runner["Eval runner (planned)"]
|
||||
R["isolated HOME · programmatic mode<br/>NDJSON capture · run manifest"]
|
||||
end
|
||||
subgraph hooks["Harness hooks (interactive work)"]
|
||||
H1["Claude Code<br/>.claude/settings.json"]
|
||||
H2["Copilot CLI<br/>.github/hooks/*.json"]
|
||||
H3["Mistral Vibe<br/>.vibe/hooks.toml"]
|
||||
H4["VS Code Chat<br/>chronicle SQLite, post hoc"]
|
||||
end
|
||||
subgraph inner["Repo layer (always on)"]
|
||||
W["wikitool emitter<br/>+ git"]
|
||||
end
|
||||
R --> T[("reports/telemetry/<session>/trace.jsonl")]
|
||||
H1 & H2 & H3 & H4 --> I["tools/trace_ingest.py"] --> T
|
||||
W --> T
|
||||
T --> S["scorers L0-L4"] --> O[("reports/evals/<date>/")]
|
||||
```
|
||||
|
||||
- **The repo layer is the truth.** `wikitool` records its own calls, so what happened *to the wiki*
|
||||
is known even when no hook fired and no runner was involved.
|
||||
- **The runner owns session boundaries.** Not every harness reports a session start - Mistral Vibe
|
||||
has no such hook - so the process that launches the agent is what brackets a run.
|
||||
- **Hooks enrich.** They add the tool calls the repo layer cannot see: file reads, greps, shell
|
||||
commands, prompts.
|
||||
|
||||
Everything joins on one session id, which every source that has to pick one resolves the same way
|
||||
(`chemenu.session`). The chain is `WIKITOOL_SESSION_ID`, then a harness's own session variable where
|
||||
one is registered (`chemenu.session.HARNESS_ENV_VARS` - Claude Code's `CLAUDE_CODE_SESSION_ID` today),
|
||||
then the parent process id. The middle step exists because the last one does not survive a harness
|
||||
that runs every tool call in its own freshly started shell. There, `os.getppid()` is a new "session"
|
||||
for every call, and neither the join nor the Iteration Budget Gate could see more than one or two
|
||||
calls of a real run. A harness only earns an entry in that chain once a live session has been
|
||||
observed setting the variable, and it has been confirmed to be the exact id the harness's own hooks
|
||||
write elsewhere in a trace. A name that merely looks plausible would mis-key a session more quietly
|
||||
than the pid fallback it replaced.
|
||||
|
||||
A trace hook that only *observes* tool calls (a `PreToolUse`/`PostToolUse`-style wiring) does not by
|
||||
itself fix a harness whose events carry a different id than `wikitool`'s own emitter. The two would
|
||||
still not join. Wiring such a hook is worth doing only once this fallback chain already keys both
|
||||
sides on the same id.
|
||||
|
||||
## The trace
|
||||
|
||||
One JSON object per line, appended to `reports/telemetry/<session>/trace.jsonl`. The contract is
|
||||
[tools/chemenu/telemetry/schema.py](../../tools/chemenu/telemetry/schema.py).
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `v` | Schema version |
|
||||
| `ts` | ISO-8601 UTC, microsecond precision |
|
||||
| `session_id` | The join key - `chemenu.session`'s fallback chain: `WIKITOOL_SESSION_ID`, else a registered harness variable, else the parent process id |
|
||||
| `pid`, `seq` | `seq` counts **within one process**. Sort a trace by `(ts, pid, seq)` |
|
||||
| `source` | `wikitool`, `runner`, or a harness name |
|
||||
| `event` | See below |
|
||||
| `attrs` | Normalised payload - the same field names whichever harness produced it |
|
||||
| `run_key` | The `work/<runkey>/` run, when one is open |
|
||||
| `trace_id`, `span_id` | From `TRACEPARENT`, when the harness exports it |
|
||||
| `redactions` | Which secret patterns fired on this event |
|
||||
|
||||
**Events.** The core - `tool.pre`, `tool.post`, `wikitool.call`, `gate.refused` - is available on
|
||||
every surface. Everything else is optional: `session.start`, `session.end`, `session.error`,
|
||||
`prompt.submitted`, `assistant.message`, `turn.end`, `tool.error`, `instructions.loaded`,
|
||||
`subagent.start`, `subagent.stop`, `compaction`, `page.written`, `publish.commit`, `budget.state` and
|
||||
`gate.cleared`.
|
||||
|
||||
**The degradation rule.** No scorer may *require* an optional event. Claude Code has about thirty
|
||||
hooks and Mistral Vibe has three. A scorer built on the rich end would silently report zero on the
|
||||
poor end, which reads as "the agent did nothing" rather than "this harness cannot say". Every
|
||||
`session.start` carries a `completeness` list naming the classes its harness can emit, so a scorer
|
||||
can answer "not measurable here" instead. What each harness can report is tabled in
|
||||
[trace-and-score-sessions.md](../how-to/trace-and-score-sessions.md#what-each-harness-can-report).
|
||||
|
||||
**Budget and trace are not the same set.** The Iteration Budget Gate exempts read-only retrieval,
|
||||
and the trace records it. What an agent looked at before acting is exactly what a trajectory scorer
|
||||
needs, and charging for a `search` would discourage the one habit that lowers cost.
|
||||
|
||||
## Whether it runs at all
|
||||
|
||||
The default depends on how the tree got here, not on one hard-coded switch.
|
||||
[tools/chemenu/telemetry/policy.py](../../tools/chemenu/telemetry/policy.py) is the one place that
|
||||
resolves it, so `wikitool doctor`, the writer and the MCP server's start-up guard all answer the
|
||||
same question the same way:
|
||||
|
||||
| Installation form | Default | Marker |
|
||||
|---|---|---|
|
||||
| Git clone of the origin repository (dev checkout) | **on** (opt-out) | No `.wikitool-release.json` |
|
||||
| An instance installed from a release, and every clone of its own repository | **off** (opt-in) | `.wikitool-release.json` present |
|
||||
|
||||
The form is read off `.wikitool-release.json`, the same stamp `version check`, `dist upgrade` and
|
||||
`version notes` already use to tell a distribution from the repository it came from. Present means
|
||||
an operator never asked for telemetry. Absent means this is the dev checkout the stack ships from,
|
||||
where the traces are the stack's own measuring instrument. An instance commits the stamp with its
|
||||
first `publish`, so a clone of it on a second machine carries the stamp and defaults to off too. It
|
||||
is a consuming instance, not a measuring stand.
|
||||
|
||||
**Turning it on for a distributed instance** takes a per-checkout `.wikitool-telemetry.json` at the
|
||||
root, gitignored like `.wikitool-remotes.json` and for the same reason. The consent to write
|
||||
prompts in cleartext to *this* disk belongs to the checkout, not the corpus, so a second clone must
|
||||
not inherit it silently. The setup instruction asks about it; nothing writes it automatically. Its
|
||||
shape is in the [configuration reference](../reference/configuration.md#telemetry).
|
||||
|
||||
**Two independent quantity caps, both enforced fail-silent in `emit()`.** Neither is enforced in
|
||||
`write_event()`, which the test suite calls directly to exercise the format without the policy
|
||||
around it.
|
||||
|
||||
- **A byte cap per session trace** (default 5 MiB), checked with one `stat` before every append.
|
||||
Once a trace is at or over the cap, further calls in that session write nothing except a single
|
||||
`telemetry.limit` event. It is elected by the same single-writer trick `session.start` uses, an
|
||||
exclusive create on a `.limit` marker file, so a trace that was cut off can be told apart from one
|
||||
whose writer simply crashed.
|
||||
- **Retention by session count** (default 250), applied once, right before a brand-new session
|
||||
directory is created. It is never applied per event, and never against the session doing the
|
||||
creating. It deletes exactly `trace.jsonl` and `.limit` from the oldest directories beyond the cap,
|
||||
and `rmdir`s a directory only once it is empty. Nothing under `reports/` is ever removed in bulk.
|
||||
|
||||
The default of 250 was set above what the origin repository's own checkout had accumulated as of
|
||||
2026-09-10: 231 session directories, well under 400 KiB in total. The cap bites on future growth,
|
||||
not on the existing history.
|
||||
|
||||
## What never reaches a trace
|
||||
|
||||
Prompts and assistant replies **are** recorded in cleartext, locally. A failure taxonomy cannot be
|
||||
read out of hashes, and building one is the first phase of any evaluation work. Four guards make
|
||||
that defensible, all of them in
|
||||
[tools/chemenu/telemetry/scrub.py](../../tools/chemenu/telemetry/scrub.py), so there is one place to
|
||||
audit:
|
||||
|
||||
1. **Secret scrubbing.** Tokens, keys, `Authorization:` headers and `SECRET=` assignments are
|
||||
replaced with `[REDACTED:<type>]`. The scrubbing is pattern-based, so it is best effort.
|
||||
2. **A content cap.** 60 KiB per attribute, with a `[TRUNCATED n chars]` marker.
|
||||
3. **A kill switch.** `WIKI_TRACE_CONTENT=0` keeps only `<field>_length` and `<field>_sha256`. The
|
||||
digest is computed either way, so traces stay comparable.
|
||||
4. **`raw/` contents never enter a trace at all.** That text is data, not instruction
|
||||
([AGENTS.md](../../AGENTS.md) invariant 4), and a trace gets read back later. Callers record a path
|
||||
and a digest.
|
||||
|
||||
Nothing leaves the machine. `reports/` is gitignored, no exporter is configured, and where a vendor
|
||||
offers one it is off. Mistral's `enable_otel` ships prompts to Mistral Studio, so it stays `false`,
|
||||
and so does `enable_telemetry`. Claude Code's and Copilot's content gates may only be enabled
|
||||
against a collector you run yourself.
|
||||
|
||||
The variables that steer all of this are in the
|
||||
[configuration reference](../reference/configuration.md#environment-variables).
|
||||
|
||||
## Evaluation levels
|
||||
|
||||
Ordered by oracle strength: hard checks first, judgment last.
|
||||
|
||||
| Level | Oracle | What it measures | Status |
|
||||
|---|---|---|---|
|
||||
| **L0** Pipeline | hard | A wiki the tools built themselves lints clean, and the catalog is a fixed point | **done** - in the origin repository's test suite |
|
||||
| **L1** Artifact scorecard | hard | Counters from `lint`'s own checks: hard errors and advisories, page count | **done** |
|
||||
| **L2** Trajectory | hard | Rules over the trace: was a refused call repeated, was a gate flag passed unearned, did a page change go unlogged | **done** |
|
||||
| **L3** Task evals | medium | Gold set: question → expected cited pages; ingest fixture → expected page titles. Scored by set overlap | needs the runner |
|
||||
| **L4** Rubric / judge | soft | Prose quality, cramming, tone | **out of scope** until a failure taxonomy exists |
|
||||
|
||||
Evaluation results are statistical, not binary. A case runs several times and reports a pass rate
|
||||
with its variance, because sampling is not deterministic. A single failure is not a merge blocker;
|
||||
a regression against a baseline is.
|
||||
|
||||
### L0 lives in the test suite, not in a separate harness
|
||||
|
||||
It runs the CLI in-process against an empty wiki. It asserts that `new` → write → `xref` →
|
||||
`index rebuild` leaves a tree `lint` calls clean, and that rebuilding the catalog again changes
|
||||
nothing on disk. That is a hard oracle over the compiler, which is what the test suite is for.
|
||||
Giving it its own runner would have duplicated the suite for nothing.
|
||||
|
||||
One behaviour it pins is easy to mistake for a defect: **a scaffolded page does not lint clean**.
|
||||
`new` writes placeholder wikilinks for the author to replace. So a page that was created but not yet
|
||||
written reports broken links, and reports its `TODO`-only sections as the advisory
|
||||
*Unfilled Template Sections*. That is the scaffold saying it is unfinished.
|
||||
|
||||
## Scoring a session
|
||||
|
||||
`tools/wikitool eval score` scores one session. It only reads, and it is exempt from the Iteration
|
||||
Budget Gate: reading back what a session did is not iterating on the wiki, and charging for it
|
||||
would discourage checking one's own work.
|
||||
|
||||
**L1** re-runs `lint`'s checks in-process and reports their counters. It shares its definition of a
|
||||
hard error with `lint --fail-on-error` - one constant, `HARD_ERROR_KEYS` - so a run can never pass
|
||||
its score while lint refuses it.
|
||||
|
||||
**L2** checks five rules. Each one restates an invariant the code cannot enforce in-process. A gate
|
||||
can refuse a call, but nothing stops an agent from calling again with the gate's own flag. That gap
|
||||
is the whole point:
|
||||
|
||||
| Rule | Invariant | Severity |
|
||||
|---|---|---|
|
||||
| `refusal-not-retried` | A refused call, repeated unchanged, is the loop the gate exists to break | error |
|
||||
| `gate-not-self-opened` | `--yes`/`-y` no longer exist at all; `--override-budget` is for a human to pass after a refusal; `--force` never | error |
|
||||
| `content-change-logged` | A publish that changes `kb/` pages needs a `log append` in the same session | advisory |
|
||||
| `clearance-was-asked-for` | A `gate.cleared` token must match one some earlier `gate.refused` issued - catches an invented token, and one reused from a different changeset | error |
|
||||
| `clearance-ended-the-turn` | No `wikitool.call` between a clearance request (exit 42) and the next `prompt.submitted` - skipped, not failed, on a harness that cannot report `prompt.submitted` | error |
|
||||
|
||||
New rules belong here only when a real trace shows a real failure. Checks invented from the contract
|
||||
text produce a score that improves while behaviour does not - the failure mode the phase-gate note
|
||||
describes. The first three rules were chosen because each guards a refusal an agent can talk its way
|
||||
around. The last two carry the load for the Mass-Update Gate's clearance mechanism (2026-08-28),
|
||||
which deliberately enforces very little in-process, as described below.
|
||||
|
||||
A rule can report a third state besides pass and fail: **skipped**, meaning this trace cannot answer
|
||||
the question at all - for example `clearance-ended-the-turn` on a harness with no prompt hook.
|
||||
Skipped reads as neither a pass nor a violation. `scorecard.render_markdown` marks it `skip`,
|
||||
distinct from `ok`, `warn` and `FAIL`, and `score()["violations"]` never includes a skipped rule,
|
||||
whatever its severity.
|
||||
|
||||
### A worked example, and where the check moved to
|
||||
|
||||
Scoring the session that shipped the telemetry layer itself used to report a failure:
|
||||
|
||||
```
|
||||
FAIL gate-not-self-opened
|
||||
call=publish --message "tools: session traces …" --yes, reason=no mass-update refusal preceded it
|
||||
```
|
||||
|
||||
That finding was true and benign at the same time. The change was 17 files, the file list was shown,
|
||||
and a human approved it. But the approval happened in conversation, and `--yes` went out on the
|
||||
first attempt, so the gate never got to refuse anything. The documented path had the refusal first,
|
||||
and the trace records tool calls, not conversations.
|
||||
|
||||
Three sessions did some version of this before the mechanism changed rather than the procedure. The
|
||||
interesting part is *which* half changed. The obvious fix was built first: make the gate impossible
|
||||
to open without independent proof of a human. That meant a ticket bound to the changeset, redeemed
|
||||
only after a `prompt.submitted` event and a `wikitool approve` typed in a second terminal. It worked,
|
||||
and it was unusable. On a harness that drives `wikitool` through a shell tool, the agent cannot
|
||||
produce the turn signal, and the human is asked to leave the conversation to approve anything at all.
|
||||
A gate nobody can pass is not a safe gate, it is an abandoned one.
|
||||
|
||||
So the enforcement got cheap and the *verification* got real:
|
||||
|
||||
- **In-process, `publish` enforces only what it can do honestly.** Exit 42 says "a human must see
|
||||
this". `--confirm <token>` says "here is what they saw", and the token digests the file list, so it
|
||||
cannot silently cover a different one. None of that proves a human was involved, and the code says
|
||||
so in as many words.
|
||||
- **The trace answers the question the tool cannot.** `clearance-ended-the-turn` looks for a
|
||||
`wikitool.call` between an exit-42 call and the next `prompt.submitted`. That is exactly the
|
||||
"refused, then confirmed in the same breath" shape all three incidents had, and it is invisible to
|
||||
any single call. `clearance-was-asked-for` catches a token that no refusal ever issued.
|
||||
|
||||
This is the loop this whole page is about, run to its conclusion: a rule that only a trace could
|
||||
check, a trace that showed it being broken three times, a mechanism rebuilt because no procedural fix
|
||||
survived - and then the check itself moved to the layer that can actually see it.
|
||||
|
||||
**What this still cannot see:** an agent that receives the exit-42 output, does not relay it, and
|
||||
passes the token anyway *after* the user happens to say something unrelated. `prompt.submitted`
|
||||
proves a turn boundary, not that the output was relayed or that the reply meant yes. That gap is
|
||||
recorded rather than papered over. Closing it needs the harness to report what the agent actually
|
||||
said, which no adapter here does yet.
|
||||
|
||||
That gap stopped being hypothetical within the hour. The first agent to receive the new gate - the
|
||||
one that had just written the paragraph above - answered the user with a file *count* and a pointer
|
||||
to "the output above". On that harness the user could not see the output: a command's stdout goes
|
||||
to the agent's context, not to anyone's screen. Nothing in the trace distinguishes that from a
|
||||
correct relay, and nothing ever will. The response was to fix the half that *is* fixable, the
|
||||
wording. The message now says "THE USER CANNOT SEE THIS OUTPUT", asks for the paths to be copied
|
||||
into the reply, and names the near-misses that do not count: a count, a summary, "the output above".
|
||||
An instruction that conflates printing with showing reads as already satisfied by the text existing.
|
||||
That is a general lesson about writing for agents, not a detail of this gate.
|
||||
|
||||
## Status
|
||||
|
||||
| Piece | State |
|
||||
|---|---|
|
||||
| Event schema, scrubber, writer | done - `tools/chemenu/telemetry/` |
|
||||
| `wikitool.call` on every command | done - one hook point in `cli.py`, next to the budget gate |
|
||||
| `gate.refused` (loop-breaker, iteration budget, mass-update) | done |
|
||||
| `publish.commit` | done |
|
||||
| Hook entry point | done - `tools/trace_ingest.py`, mappings for all three hook-capable harnesses |
|
||||
| Copilot CLI hooks | done - `.github/hooks/wiki-trace.json` |
|
||||
| VS Code chronicle import | done - `tools/import_chronicle.py` |
|
||||
| Mistral Vibe hooks | done - `.vibe/hooks.toml`, `.vibe/config.toml` |
|
||||
| Session header seeded on every trace | done - a trace declares its own `completeness` even without a session hook |
|
||||
| Scoring L1 + L2 | done - `wikitool eval score` |
|
||||
| L0 pipeline check | done - in the test suite |
|
||||
| Claude Code hooks | **partial** (2026-08-28) - `.claude/settings.json` wires `UserPromptSubmit` (the turn boundary `clearance-ended-the-turn` scores against) plus a `permissions.ask` rule on the `--confirm` form of `publish`; `PreToolUse` verified unable to force a prompt over `permissions.allow` |
|
||||
| Claude Code OTel | not started; not yet verified against the live CLI |
|
||||
| Agent runner (`eval run`) and L3 | designed below, not built |
|
||||
|
||||
## The agent runner, and why it is not here yet
|
||||
|
||||
Scoring reads what a session left behind. The missing half is *starting* one: a runner that launches
|
||||
a harness against a task, several times, and reports a pass rate. What it has to do is no longer
|
||||
guesswork, because the adapter work settled most of it:
|
||||
|
||||
- **Give each run a copy of the repository, not a fixture directory.** `config.ROOT` is derived from
|
||||
`chemenu/config.py`'s own location, so `wikitool` cannot be aimed at another tree from outside. An
|
||||
agent case therefore needs a worktree or a clone, where the tool sits inside the tree it edits. (A
|
||||
*fixture* directory works fine in-process, which is why L0 is a test.)
|
||||
- **Give each run its own HOME:** `VIBE_HOME`, `COPILOT_HOME`, plus `WIKI_TRACE_DIR` and
|
||||
`WIKITOOL_SESSION_ID`, so that state, trust decisions and traces cannot leak between runs.
|
||||
- **Never launch Vibe without `--agent`.** Its programmatic mode falls back to `auto-approve` when no
|
||||
agent is named, and it does not ask about folder trust. The runner must refuse rather than inherit
|
||||
that default.
|
||||
- **Record a run manifest:** harness and version, model, git SHA, and hashes of `AGENTS.md`, the
|
||||
published skills and the fixture. Without it, a pass rate cannot be compared to anything.
|
||||
|
||||
What is missing is not the design but the ability to check it. No provider credentials were
|
||||
configured on the machine the design was worked out on, so a live run could not be executed, let
|
||||
alone verified. Writing the runner anyway would repeat exactly the mistake the Vibe adapter avoided:
|
||||
the hook format there was wrong in the documentation, and only the installed CLI showed it. A runner
|
||||
written against an unverified mental model of three harnesses would be worse.
|
||||
|
||||
Open questions: whether Claude Code has a redirectable config directory (only `--settings` is
|
||||
documented), and whether the scrubber's own patterns are enough, or a dedicated secret scanner
|
||||
belongs in the verification step.
|
||||
+21
-3
@@ -6,6 +6,24 @@ handed to the agent. A how-to guide never retells an agent's procedure: the step
|
||||
live in `instructions/`, and a guide here says what to prepare, what to ask for and where the
|
||||
agent stops.
|
||||
|
||||
Nothing is here yet. Until the guides are written, the task-shaped parts of
|
||||
[README.md](../../README.md), [INSTALL.md](../../INSTALL.md),
|
||||
[INSTALL-MCP.md](../../INSTALL-MCP.md) and [EVALS.md](../../EVALS.md) still carry this material.
|
||||
Working with the wiki:
|
||||
|
||||
- [Ingest a source](ingest-a-source.md) - a file, a folder, a web page, or a document submitted
|
||||
through the MCP server
|
||||
- [Capture documentation from a git repository](capture-repository-docs.md) - and keep it current
|
||||
- [Export guidelines to captured repositories](export-guidelines.md)
|
||||
- [Query the wiki](query-the-wiki.md) - asking, and searching yourself
|
||||
- [Maintain the wiki](maintain-the-wiki.md) - lint, provenance, and the gates on the way to a publish
|
||||
- [Run the weekly review](run-the-weekly-review.md) - commitments in a task tracker against the
|
||||
project pages
|
||||
- [Browse the wiki in Obsidian](browse-with-obsidian.md) - and how pages are named
|
||||
|
||||
Running the instance:
|
||||
|
||||
- [Update an instance](update-an-instance.md) - to a newer release of the stack
|
||||
- [Troubleshoot an instance](troubleshoot.md)
|
||||
- [Set up the MCP read server](set-up-the-mcp-server.md) - for a consumer that is not a terminal
|
||||
on this machine
|
||||
- [Trace and score agent sessions](trace-and-score-sessions.md)
|
||||
|
||||
Installing a new instance is [INSTALL.md](../../INSTALL.md) at the repository root.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Browse the wiki in Obsidian
|
||||
|
||||
The wiki is a folder of Markdown files joined by `[[wikilinks]]`, so any Markdown editor can read
|
||||
it. Obsidian adds a graph of the links between pages, which makes it a good way to look around.
|
||||
|
||||
## What you prepare
|
||||
|
||||
Open the instance's root directory as an Obsidian vault. Obsidian keeps its own settings in
|
||||
`.obsidian/`. That folder is personal state: it is never shipped with a release, and it has no
|
||||
effect on the wiki.
|
||||
|
||||
What you get:
|
||||
|
||||
- the graph view of how pages connect;
|
||||
- navigation by wikilink, the way the agent writes them;
|
||||
- plugins such as Dataview, which can query page frontmatter, or Marp for slides.
|
||||
|
||||
## Reading, not writing
|
||||
|
||||
Browse and read freely. Change pages through the agent, though, not by editing them in Obsidian. A
|
||||
page carries generated frontmatter, cross-references and footnote citations that `tools/wikitool`
|
||||
keeps consistent across the whole wiki. A hand edit can quietly break them. Renaming or moving a
|
||||
file in Obsidian is the clearest case: a title is the page's only identifier, so it also appears in
|
||||
every wikilink and citation that points at the page. A rename goes through
|
||||
`tools/wikitool rename`, which updates every reference. The full procedure is
|
||||
[instructions/page-lifecycle.md](../../instructions/page-lifecycle.md).
|
||||
|
||||
Generated files - `kb/index.md`, `kb/log.md`, `kb/provenance.md` and every `INDEX.md` - are
|
||||
rebuilt by the tool and never edited by hand.
|
||||
|
||||
## How pages are named
|
||||
|
||||
Knowing the naming rules makes the vault easier to find your way in:
|
||||
|
||||
- **Titles are human-readable names with spaces:** `Hybrid Search.md`, not kebab-case. A wikilink
|
||||
matches the file name exactly: `[[Hybrid Search]]`.
|
||||
- **Entities are singular:** `HA Integration.md`, not `HA Integrations.md`.
|
||||
- **A title follows the subject's own established name, not the wiki's language.** `Act Runner`
|
||||
and `GitOps Ownership Model` keep their names in a German wiki too. Translating a title is a
|
||||
rename, not an edit.
|
||||
- **A title works on every operating system.** It contains none of `< > : " / \ | ? *`, is not a
|
||||
reserved name such as `CON` or `Index`, does not end in a dot, and does not differ from another
|
||||
page's title by case alone. A file's whole path below the instance root stays at 160 characters
|
||||
or fewer, so a Windows checkout works without long paths. `wikitool new` and `wikitool rename`
|
||||
refuse titles that break these rules, and `wikitool lint` reports existing ones.
|
||||
[kb/CONTRACT.md](../../kb/CONTRACT.md) has the full rule.
|
||||
|
||||
## Letting the structure grow
|
||||
|
||||
- Start with a few broad categories and refine as the material arrives. Don't over-organise early.
|
||||
- Use tags for concerns that cut across collections.
|
||||
- Review what the agent changed from time to time - `kb/log.md` lists every operation - and tell it
|
||||
when something is off. It keeps to the standards in `AGENTS.md` and the contracts, but you are the
|
||||
one who notices when a page misses the point.
|
||||
@@ -0,0 +1,59 @@
|
||||
# Capture documentation from a git repository
|
||||
|
||||
Documentation that lives in a git repository is captured, not copied by hand. A capture records the
|
||||
repository, the ref rule and the commit it came from. Later, the wiki can tell which captured
|
||||
material has fallen behind its repository, and take in the new edition as a whole.
|
||||
|
||||
## What you prepare
|
||||
|
||||
Access to the repository from this machine. Git uses your own keys and credential helpers, and
|
||||
nothing is stored in the instance. Then decide three things:
|
||||
|
||||
- the repository URL;
|
||||
- the ref to follow, usually a branch such as `main`;
|
||||
- which files count, as a glob, for example `docs/**/*.md`.
|
||||
|
||||
## Taking a repository in for the first time
|
||||
|
||||
You can run the capture yourself, or name the three values to the agent and let it run the
|
||||
command:
|
||||
|
||||
```bash
|
||||
tools/wikitool raw capture <repo-url> --ref main --path 'docs/**/*.md' --name <bundle>
|
||||
```
|
||||
|
||||
This writes the matching files, byte for byte, into `incoming/<bundle>/`, together with a
|
||||
`_capture.json` manifest naming the repository, the ref rule and the commit. From there it is an
|
||||
ordinary source: `Ingest incoming/<bundle>` ([Ingest a source](ingest-a-source.md)), with the same
|
||||
questions and the same gate.
|
||||
|
||||
## Keeping it current
|
||||
|
||||
```bash
|
||||
tools/wikitool raw status
|
||||
```
|
||||
|
||||
This lists every captured bundle that has fallen behind its repository, file by file, marking each
|
||||
file as added, modified or deleted. It only reads.
|
||||
|
||||
To bring them in, say:
|
||||
|
||||
```text
|
||||
Update the captured repositories
|
||||
```
|
||||
|
||||
The agent checks every captured bundle. It takes one changed bundle per run into the wiki, replacing
|
||||
the old edition as a whole, and tells you how many are still behind. Run it again for the next one.
|
||||
`raw capture --update` and `raw accept --replaces-bundle` are the commands underneath. What each one
|
||||
does is in the [command reference](../reference/commands/raw-material-and-uploads.md).
|
||||
|
||||
## Where the agent stops
|
||||
|
||||
The same places as any ingest: the `fidelity`/`authority` questions, the discussion of what
|
||||
matters, and the Mass-Update Gate when it publishes. When one repository cannot be reached, the
|
||||
agent reports the error and carries on with the others. A failure in one never stops the run.
|
||||
|
||||
## Related
|
||||
|
||||
- [Export guidelines](export-guidelines.md) - sending the wiki's guidelines back into the same
|
||||
repositories.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Export guidelines to captured repositories
|
||||
|
||||
The repositories the wiki captures documentation from can get something back: the wiki's
|
||||
guidelines, as one generated `GUIDELINES.md` in each repository's root, which that repository's own
|
||||
agents then read.
|
||||
|
||||
## What you prepare
|
||||
|
||||
- **Decide which pages count as guidelines.** That decision is yours, and it is written down in
|
||||
`kb/CONVENTIONS.md` § Guidelines for other repositories. Usually it is a tag. To see the file
|
||||
exactly as it would be written, without touching any repository:
|
||||
|
||||
```bash
|
||||
tools/wikitool export guidelines --tag guideline
|
||||
```
|
||||
|
||||
- **Opt each repository in, by hand, once.** A repository takes part only if it carries a
|
||||
`GUIDELINES.md` whose first line is exactly:
|
||||
|
||||
```markdown
|
||||
<!-- wikitool:export kind=guidelines -->
|
||||
```
|
||||
|
||||
Commit that one line there. Then point the repository's own `AGENTS.md` at the file (for Claude
|
||||
Code: `@GUIDELINES.md`). A `GUIDELINES.md` without that first line is hand-written, and the export
|
||||
never touches it.
|
||||
|
||||
The repository has to be one the wiki has captured
|
||||
([Capture documentation from a git repository](capture-repository-docs.md)). That is how the export
|
||||
knows where the repository lives. Its capture also has to follow a branch: a repository captured by
|
||||
a tag pattern is skipped, because a tag has no branch to commit onto.
|
||||
|
||||
## What you say
|
||||
|
||||
```text
|
||||
Roll out the guidelines to the captured repositories
|
||||
```
|
||||
|
||||
## Where the agent stops
|
||||
|
||||
The agent runs `export guidelines --push`, which stops before pushing anything. This is the
|
||||
**Guideline Push Gate** (exit 42). For every repository, it prints the status, the diff it would
|
||||
write and a `--confirm <token>` line. The agent puts every diff in front of you. Only your approval
|
||||
pushes, and then only one commit per repository, which changes only `GUIDELINES.md` and goes
|
||||
straight onto its branch.
|
||||
|
||||
The token is a digest of each target's branch tip and of the generated file. If a branch has moved
|
||||
or a guideline was edited since you looked, the gate asks again rather than pushing something you
|
||||
did not see.
|
||||
|
||||
## Related
|
||||
|
||||
- [Guideline export commands](../reference/commands/guideline-export.md)
|
||||
- [Why the gates are code](../explanation/why-gates-are-code.md)
|
||||
@@ -0,0 +1,82 @@
|
||||
# Ingest a source
|
||||
|
||||
Turn a file, a folder or a web page into wiki pages: a source page that summarises it, and the
|
||||
entity and concept pages it touches, cross-referenced and published. The agent does the work, using
|
||||
the `wiki-ingest` skill ([skill catalogue](../reference/skills.md)). This page covers what you hand
|
||||
it and where it waits for you.
|
||||
|
||||
## What you prepare
|
||||
|
||||
Put the material into `incoming/` at the root of the instance. You don't have to classify it or
|
||||
pick a directory:
|
||||
|
||||
- **One file** goes in directly: `incoming/my-article.md`.
|
||||
- **Several files that belong together** go into one folder, `incoming/<name>/`. The folder is one
|
||||
source. It is accepted whole, and its structure is kept.
|
||||
- **A web page** needs no download of your own. Name the URL to the agent, and
|
||||
`tools/wikitool raw fetch` puts two files into `incoming/`: the HTML exactly as received, and a
|
||||
text derived from it with a header recording where and when it was fetched.
|
||||
- **A page behind a paywall or a login** is saved from your browser into `incoming/`, HTML only. The
|
||||
agent derives the same text from that file with `raw fetch --html`.
|
||||
|
||||
Where a file ends up in `raw/` is computed by `tools/wikitool raw accept`: a `YYYY/MM` shard for the
|
||||
day it was accepted. So is whether several files of one source get bundled. You never choose either.
|
||||
`incoming/` itself is gitignored, so whatever waits there is never published by accident.
|
||||
|
||||
## What you say
|
||||
|
||||
```text
|
||||
Ingest incoming/my-article.md
|
||||
Ingest https://example.org/post
|
||||
Ingest
|
||||
```
|
||||
|
||||
A bare `Ingest` takes the oldest entry waiting in `incoming/`. `tools/wikitool raw pending` lists
|
||||
what is waiting and names the entry the next bare `Ingest` will take.
|
||||
|
||||
## Where the agent asks you
|
||||
|
||||
- **Before promoting the file into `raw/`**, the agent asks how faithful the capture is
|
||||
(`fidelity`) and what the material may claim about its subject (`authority`). Both are recorded
|
||||
once and never guessed: you know the answers now, and nobody can recover them later.
|
||||
[raw/CONTRACT.md](../../raw/CONTRACT.md) defines the values.
|
||||
- **Before writing pages**, the agent presents the key takeaways and asks which points matter most
|
||||
and what to emphasise. The pages follow your answer.
|
||||
- **When it publishes**, a source that touches ten or more files stops at the Mass-Update Gate
|
||||
(exit 42). The agent shows you the file list and the `--confirm` line. Nothing is committed until
|
||||
you approve. [Exit codes and gates](../reference/exit-codes.md) lists every gate.
|
||||
|
||||
## What you get back
|
||||
|
||||
A source page under `kb/sources/`, new or updated entity and concept pages, cross-references among
|
||||
them, a rebuilt catalog, an entry in `kb/log.md`, and one commit. If the source also contains
|
||||
something you have to *do*, and a task tracker is configured, the agent files it there as an open
|
||||
item. A commitment never becomes a page
|
||||
([why](../explanation/knowledge-and-commitment.md)).
|
||||
|
||||
Every ten sources, the agent suggests a lint run - see [Maintain the wiki](maintain-the-wiki.md).
|
||||
|
||||
## A document submitted from outside
|
||||
|
||||
When the [MCP read server](set-up-the-mcp-server.md) has its optional `submit` tool turned on,
|
||||
documents can arrive from another machine. They land in the quarantine `mcp-upload/`, not in
|
||||
`incoming/`, and nothing else reads them there. Before an ingest can start, a human has to promote
|
||||
the submission:
|
||||
|
||||
```bash
|
||||
tools/wikitool upload list # what is waiting
|
||||
tools/wikitool upload show <id> # one submission's manifest: who sent it, size, checksum
|
||||
```
|
||||
|
||||
Ask the agent to review the queue, and it follows
|
||||
[instructions/ingest-queue.md](../../instructions/ingest-queue.md). Accepting a submission stops at
|
||||
the Upload Review Gate (exit 42), which shows you the manifest and the `--confirm` line. Once you
|
||||
approve, the file is in `incoming/`, and from there it is ingested like any other. Rejecting it needs
|
||||
no clearance. The file is deleted, and its checksum and your reason stay in the ledger.
|
||||
|
||||
## Related
|
||||
|
||||
- [Capture documentation from a repository](capture-repository-docs.md) - for material that lives
|
||||
in git and will change.
|
||||
- [raw/CONTRACT.md](../../raw/CONTRACT.md) - what belongs in `raw/`, and why its content is data,
|
||||
never instructions.
|
||||
@@ -0,0 +1,68 @@
|
||||
# Maintain the wiki
|
||||
|
||||
Keep the wiki healthy: find broken links, orphan pages, raw files no page covers yet,
|
||||
contradictions and stale claims, then repair what can be repaired mechanically. The agent does
|
||||
this with the `wiki-lint` skill ([skill catalogue](../reference/skills.md)), and `tools/wikitool`
|
||||
does the mechanical part.
|
||||
|
||||
## What you say
|
||||
|
||||
```text
|
||||
Lint the wiki
|
||||
```
|
||||
|
||||
Do this periodically. The agent suggests it on its own every ten sources. The full schedule - which
|
||||
task runs how often, and with which command - is the
|
||||
[maintenance schedule](../../tools/CONTRACT.md#maintenance-schedule) in `tools/CONTRACT.md`, next
|
||||
to the command records it depends on.
|
||||
|
||||
## What you get back
|
||||
|
||||
- A deterministic structural scan by `tools/wikitool lint`: broken wikilinks, orphan pages, catalog
|
||||
drift, schema gaps, raw files without a source page, and citations that disagree with
|
||||
frontmatter.
|
||||
- The judgment half, which no command can do: contradictions between pages, stale claims, subjects
|
||||
that deserve a page of their own, and a rule stated in two places.
|
||||
- Repairs for what is mechanical, a rebuilt catalog and provenance index, and an entry in
|
||||
`kb/log.md`.
|
||||
- A report under `reports/`, which is gitignored and never published. The agent carries the
|
||||
findings that need you into its reply.
|
||||
|
||||
For a quick look without a full pass, say `Wiki status` instead. That is read-only.
|
||||
|
||||
## Provenance: where a claim came from
|
||||
|
||||
Every fact should trace back to a raw file. A source page lists its raw files, and other pages
|
||||
cite specific claims inline with a footnote that points at a source page. You can ask the
|
||||
questions yourself:
|
||||
|
||||
```bash
|
||||
tools/wikitool sources coverage # raw files no source page covers yet
|
||||
tools/wikitool sources trace --raw raw/2026/09/x.md # which pages rest on this file
|
||||
tools/wikitool sources trace --page "HA Integration" # which raw files this page rests on
|
||||
```
|
||||
|
||||
`kb/provenance.md` is the generated reverse index. The commands are in the
|
||||
[provenance reference](../reference/commands/provenance.md).
|
||||
|
||||
## Where the agent stops
|
||||
|
||||
- **Publishing ten or more files** stops at the Mass-Update Gate (exit 42). A lint pass that repairs
|
||||
many pages does this routinely. The agent shows you the full file list and the `--confirm` line,
|
||||
and publishes only after you approve. The token is a digest of that file list, so your approval
|
||||
never covers a changeset you did not see.
|
||||
- **The call budget.** Every `wikitool` call counts against a per-session budget: 60 calls, or 3
|
||||
identical calls in a row. Past it, calls are refused. The agent stops and summarises where it is.
|
||||
If you want it to go on, tell it so. Only then may it pass `--override-budget` on the command it
|
||||
needs. A refusal usually means the task should be split, not pushed through.
|
||||
- **No reachable remote.** Without `--no-push`, `publish` refuses with exit 1 before committing when
|
||||
the remote is not configured or cannot be reached. An instance that is meant to stay local passes
|
||||
`--no-push` on every publish.
|
||||
|
||||
Every gate, and which command it belongs to: [exit codes and gates](../reference/exit-codes.md).
|
||||
Why they are enforced in code: [why-gates-are-code.md](../explanation/why-gates-are-code.md).
|
||||
|
||||
## Related
|
||||
|
||||
- [Troubleshoot](troubleshoot.md) - when `doctor` or a command reports a problem with the instance
|
||||
itself rather than with its pages.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Query the wiki
|
||||
|
||||
Ask the wiki a question and get an answer built from its compiled pages, with the pages and sources
|
||||
it came from. The agent answers using the `wiki-query` skill
|
||||
([skill catalogue](../reference/skills.md)), which only reads unless you ask it to keep an answer.
|
||||
|
||||
## What you say
|
||||
|
||||
Ask the way you would ask a colleague:
|
||||
|
||||
```text
|
||||
What projects use MQTT?
|
||||
Show me the architecture of HA Integration
|
||||
Compare gdeploy and plugnburn-edl
|
||||
What decisions were made about E3DC integration?
|
||||
```
|
||||
|
||||
## What you get back
|
||||
|
||||
An answer that names the wiki pages it came from and the sources behind them. Where the wiki has no
|
||||
source for a claim, the agent says so instead of filling the gap. An answer it cannot back is never
|
||||
written into the wiki either.
|
||||
|
||||
If an answer is worth keeping - a comparison you will want again, a synthesis across several pages -
|
||||
the agent proposes filing it back as a page. It names the pages it would create before it creates
|
||||
any, and nothing is written unless you agree.
|
||||
|
||||
## Searching yourself
|
||||
|
||||
The agent searches with the same command you can run:
|
||||
|
||||
```bash
|
||||
tools/wikitool search "<text>"
|
||||
tools/wikitool search --field entity_type=system
|
||||
tools/wikitool search --field '!sources' # pages without a sources: list
|
||||
```
|
||||
|
||||
`search` covers all of `kb/`, by text, by frontmatter or both. Each hit carries the page's path and
|
||||
full title, so you can open it directly. It only reads, and it does not count against the
|
||||
session's call budget, because looking before acting is the habit worth encouraging. The predicate forms are in the
|
||||
[command reference](../reference/commands/finding-and-checking.md).
|
||||
|
||||
For browsing rather than searching, `kb/index.md` is the catalog map, and each collection carries
|
||||
its own `INDEX.md`. All of these are generated. Read them, but never edit them. To browse the pages as
|
||||
a graph, see [Browse the wiki in Obsidian](browse-with-obsidian.md).
|
||||
|
||||
## Asking from another machine
|
||||
|
||||
A consumer that is not a terminal on this machine can ask the same questions through the MCP read
|
||||
server and get the same answers: [Set up the MCP read server](set-up-the-mcp-server.md).
|
||||
@@ -0,0 +1,64 @@
|
||||
# Run the weekly review
|
||||
|
||||
Go through your committed initiatives once a week and decide what each one needs: a next action, a
|
||||
pause, a close. A machine prepares the list, so you don't have to reconstruct it from memory. The
|
||||
agent runs the `gtd-weekly-review` skill ([skill catalogue](../reference/skills.md)).
|
||||
|
||||
## Two places, on purpose
|
||||
|
||||
Knowledge and commitments change at different speeds, so they live in different places:
|
||||
|
||||
- **A page under `kb/gtd/`** is one initiative's durable memory: its goal, who is involved, where it
|
||||
stands, and why it is worth doing. It never summarises the task list.
|
||||
- **The open items** live in a task tracker, which owns them.
|
||||
|
||||
Nothing syncs between the two. `tools/wikitool review` joins them when it reads, matching on the
|
||||
project name, and stores nothing - not even a report file. Why the split runs this way rather than
|
||||
syncing: [knowledge-and-commitment.md](../explanation/knowledge-and-commitment.md).
|
||||
|
||||
## What you prepare
|
||||
|
||||
A task tracker, configured for this checkout in `.wikitool-tasks.json`. Super Productivity and
|
||||
CalDAV (for example Nextcloud Tasks) are supported. The file's shape, and the difference between a
|
||||
tracker the wiki can only read and one it can write to, is in the
|
||||
[configuration reference](../reference/configuration.md#task-tracker). Without a tracker, the
|
||||
review has nothing to join against. That is a valid state, though, and the pages work without one.
|
||||
|
||||
## What you say
|
||||
|
||||
```text
|
||||
Run the weekly review
|
||||
```
|
||||
|
||||
## What you get back
|
||||
|
||||
The agent runs `tools/wikitool review` and walks you through its findings, one at a time:
|
||||
|
||||
- initiatives with no next action;
|
||||
- waiting-fors past their follow-up date;
|
||||
- tracker projects that have no page;
|
||||
- active pages with no open item;
|
||||
- someday items that have gone stale.
|
||||
|
||||
For each finding, the agent asks what is actually true, because "this looks stalled" can mean a
|
||||
next action is missing, the initiative was paused on purpose, or it is finished. Then it proposes the
|
||||
exact change: a new item in the tracker, an item marked done, or a page's state moved to `dormant`,
|
||||
`completed` or `abandoned`. Nothing is written to the tracker until you confirm that exact call.
|
||||
What was decided goes onto the page, and the page is published.
|
||||
|
||||
A quiet week is a valid result. If one of the checks could not run because the tracker did not
|
||||
answer, the agent says which check was skipped. An empty result under a skipped check means nothing.
|
||||
|
||||
## Starting something new
|
||||
|
||||
Ask the agent for a new project and name it. It runs `tools/wikitool new project`, which gives the
|
||||
initiative its page under `kb/gtd/` and, where a tracker is configured, a tracker project under the
|
||||
same name. A CalDAV tracker gets that project created automatically. On Super Productivity the
|
||||
command stops with exit 42 and asks you to create the project in the app yourself. Tell the agent
|
||||
when you are done, and it continues with `--resume`. A tracker the wiki can only read refuses the
|
||||
command outright.
|
||||
|
||||
A single open item with no page of its own - a commitment that came with a source, say - is filed
|
||||
with `tools/wikitool task new`. `task list` reads a project's open items back, and `task close --id`
|
||||
marks one done. Nothing deletes an item. The commands are in the
|
||||
[command reference](../reference/commands/pages.md).
|
||||
@@ -0,0 +1,263 @@
|
||||
# Set up the MCP read server
|
||||
|
||||
Serve an instance's wiki read-only over MCP, so that a consumer that is not a terminal on this
|
||||
machine - another agent, a chat client, a service - can ask it the same questions and get the same
|
||||
answers. This page covers what you set up around the server: the client, the authentication in
|
||||
front of it, the optional `submit` path, and how to check that it works. Installing, starting and
|
||||
syncing the server itself is
|
||||
[instructions/mcp-read-server.md](../../instructions/mcp-read-server.md), which an agent follows
|
||||
and which is readable for you as well.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [What the server is, and what it is not](#what-the-server-is-and-what-it-is-not)
|
||||
- [What you prepare](#what-you-prepare)
|
||||
- [Wiring a client to it](#wiring-a-client-to-it)
|
||||
- [Running it as a service](#running-it-as-a-service)
|
||||
- [Authentication in front of it](#authentication-in-front-of-it)
|
||||
- [Keeping the served tree current](#keeping-the-served-tree-current)
|
||||
- [Optional: the submit path](#optional-the-submit-path)
|
||||
- [Checking that it works](#checking-that-it-works)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [What is deliberately not here](#what-is-deliberately-not-here)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## What the server is, and what it is not
|
||||
|
||||
The server is a second consumer of the same core as the CLI, not a second program. Both call the
|
||||
same functions, and a golden test in the origin repository holds their output together. What
|
||||
`tools/wikitool search --json` returns, the MCP tool `search` returns too, together with the commit
|
||||
the answer was computed from.
|
||||
|
||||
It is not a write path into `kb/`. No tool creates, changes or publishes a page - not because a list
|
||||
is filtered, but because the server imports nothing under `tools/chemenu/commands/`. Those functions
|
||||
cannot be reached from its process. The one exception is opt-in: a sixth tool, `submit`, which
|
||||
writes only into a quarantine no other command reads. A human decides about everything that leaves
|
||||
it ([below](#optional-the-submit-path)).
|
||||
|
||||
| Tool | What it answers |
|
||||
|---|---|
|
||||
| `search` | Pages in `kb/` by text, by frontmatter (`!sources`, `tags~k8s`), or both |
|
||||
| `types` | Which page types this wiki knows |
|
||||
| `describe_type` | A type's full contract: fields, required values, enums |
|
||||
| `lint` | Structural findings: broken wikilinks, orphans, catalog drift, schema gaps |
|
||||
| `status` | A snapshot: page count, distribution over collections, finding counts |
|
||||
| `submit` *(optional)* | Hands a document to the review queue - into a quarantine, never into `kb/` |
|
||||
|
||||
## What you prepare
|
||||
|
||||
- A working instance ([INSTALL.md](../../INSTALL.md)), including `tools/.venv` and ripgrep.
|
||||
- For a deployment, a reverse proxy that can authenticate requests before they reach the server
|
||||
([below](#authentication-in-front-of-it)).
|
||||
|
||||
Then ask the agent:
|
||||
|
||||
```text
|
||||
Set up the MCP read server for this instance
|
||||
```
|
||||
|
||||
It installs the server's optional dependency, which is deliberately not in
|
||||
`tools/requirements.txt`: an instance that only uses the CLI should not have to install pydantic,
|
||||
starlette, uvicorn and cryptography. It also checks that tracing cannot write into the served tree.
|
||||
The server refuses to start if it could:
|
||||
|
||||
```text
|
||||
ERROR Telemetry is on and would write into the served checkout (...). Set WIKI_TRACE=0,
|
||||
or point WIKI_TRACE_DIR outside the corpus.
|
||||
```
|
||||
|
||||
That refusal is expected, not an installation fault. Telemetry is off by default in an instance
|
||||
installed from a release and on in a clone of the origin repository
|
||||
([configuration](../reference/configuration.md#telemetry)). The server consults the same policy
|
||||
`wikitool doctor` does. Setting `WIKI_TRACE=0` explicitly, as every example here does, is correct
|
||||
either way and makes the check moot.
|
||||
|
||||
## Wiring a client to it
|
||||
|
||||
Locally, over `stdio`, the client starts the server itself: one process per consumer, no network.
|
||||
The configuration follows the usual MCP client shape, with absolute paths, because the client
|
||||
inherits no working directory:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"chemenu": {
|
||||
"command": "/path/to/instance/tools/.venv/bin/python",
|
||||
"args": ["-m", "chemenu.mcp"],
|
||||
"cwd": "/path/to/instance/tools",
|
||||
"env": {
|
||||
"WIKI_TRACE": "0",
|
||||
"CHEMENU_ROOT": "/path/to/instance"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`CHEMENU_ROOT` says **which** wiki is served. Without it, the server takes the checkout its own
|
||||
package sits in. That is enough for a single instance, but with several corpora it is safer to always
|
||||
set it. The process speaks MCP over stdin and stdout and prints nothing of its own, which is correct.
|
||||
|
||||
## Running it as a service
|
||||
|
||||
Deployed, the server speaks `streamable-http`, the only transport a reverse proxy can sit in front
|
||||
of. The command a service unit or container runs looks like this:
|
||||
|
||||
```bash
|
||||
WIKI_TRACE=0 CHEMENU_ROOT=/srv/chemenu \
|
||||
tools/.venv/bin/python -m chemenu.mcp \
|
||||
--transport streamable-http --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
The endpoint is then `http://<host>:8000/mcp`. `--host 0.0.0.0` is deliberately not the default:
|
||||
without it the server binds to loopback, which is right locally and wrong in a container, where the
|
||||
proxy has to reach it. If you set it, make sure something stands in front. `sse` is not offered. It
|
||||
is the superseded remote transport, and building on it now would only postpone the switch.
|
||||
|
||||
## Authentication in front of it
|
||||
|
||||
**The server does not authenticate on its own, by design.** A request that is not properly
|
||||
authenticated should never reach the Python process at all. The authentication is a Traefik
|
||||
ForwardAuth middleware:
|
||||
|
||||
> **<https://gitea.nehmer.net/torben/gitea-mcp-forward-auth>**
|
||||
|
||||
In short, it checks `Authorization: Bearer <token>` against SHA-256 hashes of allowed tokens. It
|
||||
answers `200` for a valid token and `401` for a missing or wrong one, and always leaves
|
||||
`GET /healthz` open. Plaintext tokens appear neither in the configuration nor in the log, only
|
||||
hashes and a short fingerprint. It is configured with `AUTH_PROXY_TOKEN_HASHES_DIR` (a directory with
|
||||
one file per token hash, which suits a Kubernetes secret volume) or `AUTH_PROXY_TOKEN_HASHES` (a
|
||||
comma-separated list). The README there has the details, reference manifests and a test script.
|
||||
|
||||
To create a token hash:
|
||||
|
||||
```bash
|
||||
echo -n "my-token" | sha256sum | awk '{print $1}'
|
||||
```
|
||||
|
||||
**Rate limiting belongs in the same place,** in front of the process and beside the
|
||||
authentication. It does not belong in the Iteration Budget Gate. That gate stops an *agent session*
|
||||
from iterating over the wiki's state unnoticed, which is why retrieval is exempt from it. Using it
|
||||
as a rate limiter would water it down.
|
||||
|
||||
## Keeping the served tree current
|
||||
|
||||
The server reads the working tree, and a stale checkout answers confidently and wrongly. So **every
|
||||
answer carries the commit** it was computed from:
|
||||
|
||||
```json
|
||||
{ "commit": "<40-character SHA>", "as_of": "<ISO-8601, UTC>", "count": 3, "...": "..." }
|
||||
```
|
||||
|
||||
A timer beside the server keeps the tree current by polling: a fetch, then a hard reset to the
|
||||
remote branch. The two commands, and why the hard reset is load-bearing, are in the instruction's
|
||||
sync step. Polling rather than a webhook, because it needs no inbound endpoint and no signature
|
||||
check, which is a smaller attack surface than the thing a webhook would optimise. **Never add
|
||||
`git clean` to that timer.** The reset leaves every gitignored path alone, which is what keeps the
|
||||
`submit` quarantine and any traces intact. A clean would silently delete a submission nobody has
|
||||
reviewed yet.
|
||||
|
||||
## Optional: the submit path
|
||||
|
||||
Without this, `submit` does not exist as a tool - not unused, but not registered. A file
|
||||
`.wikitool-upload.json` in the served corpus turns it on:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": 1,
|
||||
"identity_header": "X-Forwarded-User",
|
||||
"max_bytes": 10485760,
|
||||
"allowed_extensions": [".md", ".txt", ".pdf", ".html", ".csv", ".json", ".png", ".jpg"],
|
||||
"quota": { "submissions_per_day": 20, "bytes_per_day": 52428800 }
|
||||
}
|
||||
```
|
||||
|
||||
Every field is required, and none has a built-in default except `identity_header`. A malformed file
|
||||
stops the server from starting. It never means "no limits": the failure deliberately goes the safe
|
||||
way. Size and quota are your deployment's decision, not defaults worth copying blindly.
|
||||
|
||||
**With `submit` on, the middleware has a second duty.** It has to set the identity header itself, on
|
||||
every authenticated request, and discard any copy the client sent. The server trusts that header as
|
||||
the value for `submitter`. A header the client could set itself would be a claim, not an identity.
|
||||
If `identity_header` names a header the middleware does not set, every submission is refused for
|
||||
lack of an identity.
|
||||
|
||||
Submitted files land in `mcp-upload/<id>/`, which is gitignored and read by no other command. From
|
||||
there a human promotes or rejects them, in the same checkout as the server rather than in its
|
||||
process. That review is part of
|
||||
[ingesting a source](ingest-a-source.md#a-document-submitted-from-outside).
|
||||
|
||||
## Checking that it works
|
||||
|
||||
The quickest test without a client starts the server over stdio, lists its tools and asks one
|
||||
question:
|
||||
|
||||
```bash
|
||||
cd tools && WIKI_TRACE=0 .venv/bin/python - <<'EOF'
|
||||
import asyncio, os
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
|
||||
async def main():
|
||||
params = StdioServerParameters(
|
||||
command=".venv/bin/python", args=["-m", "chemenu.mcp"],
|
||||
env={"WIKI_TRACE": "0", "PATH": os.environ["PATH"]},
|
||||
)
|
||||
async with stdio_client(params) as (r, w):
|
||||
async with ClientSession(r, w) as s:
|
||||
await s.initialize()
|
||||
print("Tools:", [t.name for t in (await s.list_tools()).tools])
|
||||
out = await s.call_tool("status", {})
|
||||
d = getattr(out, "structuredContent", None) or out.structured_content
|
||||
print("Pages:", d["pages"], "| Commit:", d["commit"])
|
||||
|
||||
asyncio.run(main())
|
||||
EOF
|
||||
```
|
||||
|
||||
Expected output, roughly:
|
||||
|
||||
```text
|
||||
Tools: ['search', 'types', 'describe_type', 'lint', 'status']
|
||||
Pages: 176 | Commit: 576df2cdddc96614a7e6641e562022d52112d411
|
||||
```
|
||||
|
||||
Then cross-check against the CLI. Both have to give the same answer:
|
||||
|
||||
```bash
|
||||
tools/wikitool search "<term>" --json
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**`ERROR Telemetry is on and would write into the served checkout`**: expected behaviour, not an
|
||||
installation fault. Set `WIKI_TRACE=0`, or point `WIKI_TRACE_DIR` outside the corpus.
|
||||
|
||||
**`ModuleNotFoundError: No module named 'mcp'`**: the optional dependency is not installed, or
|
||||
the client starts a different Python than the instance's. Give the client the absolute path to
|
||||
`tools/.venv/bin/python`.
|
||||
|
||||
**`"commit": null` in every answer**: the served tree has uncommitted changes. Either the sync is not
|
||||
running, or something writes into the corpus that has no business there. The five read tools never
|
||||
write. `submit`, when it is on, writes only to `mcp-upload/`, which is gitignored and so is never a
|
||||
cause of `"commit": null` either.
|
||||
|
||||
**`commit` names an old revision**: the sync timer is not running.
|
||||
|
||||
**The server answers differently from `wikitool`**: that is a defect, not a configuration
|
||||
difference, since both go through the same functions. First check that both point at the same root.
|
||||
`CHEMENU_ROOT` is easy to set for one and not the other.
|
||||
|
||||
**Not reachable from outside**: without `--host 0.0.0.0` the server binds to loopback. If it is
|
||||
reachable but every call gets `401`, the middleware is working correctly and the token is wrong.
|
||||
|
||||
## What is deliberately not here
|
||||
|
||||
Deployment specifics - the cluster, ingress hosts, the secret store, the GitOps source - are
|
||||
private infrastructure, and this repository is public.
|
||||
|
||||
There is no container image for the server yet. Until there is, the path above - the venv,
|
||||
`python -m chemenu.mcp`, a proxy in front - is the complete one. With `submit` on, `mcp-upload/`
|
||||
has to survive restarts the same way the rest of the checkout does. Otherwise a submitted file that
|
||||
nobody has reviewed yet loses its quarantine.
|
||||
@@ -0,0 +1,176 @@
|
||||
# Trace and score agent sessions
|
||||
|
||||
Record what an agent did in a session, read the record back, and score it. A score covers the state
|
||||
the session left the wiki in and how it got there: whether it repeated a refused call, or passed a
|
||||
gate's flag without that gate ever refusing. Why this exists, what the trace contains, and what a
|
||||
score means are covered in [telemetry-and-evaluation.md](../explanation/telemetry-and-evaluation.md).
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [What you prepare](#what-you-prepare)
|
||||
- [Reading a session back](#reading-a-session-back)
|
||||
- [What each harness can report](#what-each-harness-can-report)
|
||||
- [Claude Code](#claude-code)
|
||||
- [Copilot CLI](#copilot-cli)
|
||||
- [VS Code Chat](#vs-code-chat)
|
||||
- [Mistral Vibe](#mistral-vibe)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## What you prepare
|
||||
|
||||
**Turn tracing on, if it is off.** It is on in a clone of the origin repository and off in an
|
||||
instance installed from a release. `tools/wikitool doctor` reports which state this checkout is in,
|
||||
and why. To turn it on for one checkout, create `.wikitool-telemetry.json` at the root:
|
||||
|
||||
```json
|
||||
{ "enabled": true }
|
||||
```
|
||||
|
||||
The file and its other keys are in the [configuration reference](../reference/configuration.md#telemetry).
|
||||
Everything stays on this machine: traces land under `reports/telemetry/`, which is gitignored, and
|
||||
no exporter is configured.
|
||||
|
||||
**Scope the session to the task.** Every source of events joins on one session id. Setting it
|
||||
yourself keeps one task in one trace:
|
||||
|
||||
```bash
|
||||
export WIKITOOL_SESSION_ID="my-task"
|
||||
```
|
||||
|
||||
[instructions/session-setup.md](../../instructions/session-setup.md) has the line for each shell,
|
||||
and what to do on a harness that starts a fresh shell for every command.
|
||||
|
||||
## Reading a session back
|
||||
|
||||
```bash
|
||||
tools/wikitool eval sessions # which sessions have a trace
|
||||
|
||||
# what did that session do?
|
||||
jq -r '[.ts, .source, .event, (.attrs.command // .attrs.tool_name // "")] | @tsv' \
|
||||
reports/telemetry/my-task/trace.jsonl
|
||||
|
||||
# how well did it do it?
|
||||
tools/wikitool eval score --session my-task
|
||||
tools/wikitool eval score --session my-task --save # also write the scorecard under reports/evals/
|
||||
```
|
||||
|
||||
Both `eval` commands only read, and neither counts against the session's call budget: reading back
|
||||
what a session did is not iterating on the wiki. Their options are in the
|
||||
[telemetry command reference](../reference/commands/telemetry.md).
|
||||
|
||||
To see how a harness hook payload would be recorded, feed one in by hand without writing anything:
|
||||
|
||||
```bash
|
||||
echo '{"session_id":"x","hook_event_name":"post_tool","tool_name":"bash"}' \
|
||||
| tools/trace_ingest.py --source mistral-vibe --dry-run
|
||||
```
|
||||
|
||||
## What each harness can report
|
||||
|
||||
`wikitool` records its own calls on every harness. What the agent did *between* those calls -
|
||||
file reads, searches, shell commands, prompts - comes from the harness's hooks, and harnesses differ
|
||||
a lot in what they offer. Verified against vendor documentation on 2026-08-23:
|
||||
|
||||
| | Claude Code | Copilot CLI | VS Code Chat | Mistral Vibe |
|
||||
|---|---|---|---|---|
|
||||
| Hook events | ~30 | 14 | none | 3 (`pre_tool`, `post_tool`, `post_agent`) |
|
||||
| Session start/end hook | yes | yes | - | **no** |
|
||||
| Prompt submit hook | yes | yes | - | **no** |
|
||||
| Which instructions loaded | yes (`InstructionsLoaded`) | no | no | no |
|
||||
| Block / rewrite a tool call | yes | yes | - | yes |
|
||||
| OTel to your own collector | yes | via MDM `telemetry` | no | **no** - `enable_otel` targets Mistral Studio only |
|
||||
| `TRACEPARENT` to subprocesses | yes | undocumented | - | undocumented |
|
||||
| Programmatic mode | `-p`, `stream-json` | `-p` | no | `-p`, `--output streaming` |
|
||||
| Isolated config home | to be confirmed | `COPILOT_HOME` | no | `VIBE_HOME` |
|
||||
| Local session log | `~/.claude/projects/*.jsonl` (unstable format) | `~/.copilot/session-state/<id>/events.jsonl` | chronicle SQLite | `$VIBE_HOME/logs/` (no format guarantee) |
|
||||
| Wired up here | **partial** - `.claude/settings.json` (`UserPromptSubmit` + a `permissions.ask` rule on the clearing publish) | **yes** - `.github/hooks/wiki-trace.json` | **yes** - `tools/import_chronicle.py` | **yes** - `.vibe/hooks.toml` |
|
||||
|
||||
A rule that needs an event a harness cannot report is shown as *skipped*, not as passed or failed.
|
||||
|
||||
### Claude Code
|
||||
|
||||
`.claude/settings.json` wires `UserPromptSubmit` to `tools/trace_ingest.py`, through
|
||||
`tools/trace-hook` like every hook here. That gives the trace its turn boundaries, which the rule
|
||||
`clearance-ended-the-turn` needs. Without a `prompt.submitted` event it reports "cannot say"
|
||||
instead of a verdict.
|
||||
|
||||
**No `PreToolUse` decision hook is wired, and that is a finding, not an oversight.** Verified against
|
||||
the live CLI on 2026-08-27: a `PreToolUse` hook that returns
|
||||
`hookSpecificOutput.permissionDecision: "ask"` does **not** override a matching `permissions.allow`
|
||||
rule. Permissions are evaluated before the hook's decision, so a hook cannot force a confirmation
|
||||
prompt on a command that is already allowed. A `permissions.ask` rule *does* win, which is why
|
||||
`.claude/settings.json` carries one on the `--confirm` form of `publish`
|
||||
(`Bash(tools/wikitool publish --confirm:*)`). The clearing call prompts, and ordinary publishes below
|
||||
the threshold do not. It is a prefix match, so it depends on `--confirm` coming first, which is why
|
||||
`publish` always prints its re-run line in that order. Treat it as a useful second line, not a
|
||||
guarantee: an agent that reorders the flags gets around it.
|
||||
|
||||
### Copilot CLI
|
||||
|
||||
[.github/hooks/wiki-trace.json](../../.github/hooks/wiki-trace.json) is committed, so a clone
|
||||
brings its own tracing. Eleven events go to `tools/trace_ingest.py`. `disableAllHooks` in
|
||||
`.github/copilot/settings.json` turns them off without deleting anything. `userPromptSubmitted` is
|
||||
among them, so `clearance-ended-the-turn` can be scored on Copilot CLI with no change.
|
||||
|
||||
A `preToolUse` entry that forces a decision on the clearing call would first need Copilot's own
|
||||
decision-document schema verified against a live CLI. That has not happened, and an adapter that
|
||||
cannot be verified is not written.
|
||||
|
||||
Four details of the hook configuration matter. The first two hold for all three harnesses that have
|
||||
hooks:
|
||||
|
||||
- **The interpreter is the venv's, never the script's shebang.** Each `bash` command is
|
||||
`./tools/trace-hook ...`, which runs `trace_ingest.py` with `tools/.venv`'s Python in either venv
|
||||
layout. Each `powershell` command names `tools\.venv\Scripts\python.exe` directly. On Windows in
|
||||
Git Bash, the shebang's `python3` was the Microsoft Store alias, which made every hook a silent
|
||||
no-op. Before the preflight has created the venv, `trace-hook` records nothing and exits 0.
|
||||
- **`./tools/trace-hook` has a PowerShell twin, `tools/trace-hook.ps1`.** PowerShell on Windows
|
||||
resolves the string to the `.ps1` first. Without the twin, it hands the sh script to a file
|
||||
association, and Windows asks which app should open it - on every hook event. Copilot CLI also
|
||||
reads `.claude/settings.json` and runs its single `command` under PowerShell on Windows, so a
|
||||
`bash` field alone does not keep the string out of PowerShell. The twin follows the sh script's
|
||||
rules - venv Python, silent without a venv, exit 0 whatever happens - and stays parseable by
|
||||
Windows PowerShell 5.1, because VS Code starts hooks under 5.1.
|
||||
- **Every command ends in `|| true`.** `preToolUse` hooks fail closed: a non-zero exit denies the
|
||||
tool call. Without the guard, a missing interpreter would turn the observer into a blocker that
|
||||
refuses every tool call in the session. Timeouts fail open, so the 5-second `timeoutSec` is no
|
||||
hazard.
|
||||
- **The event name is passed explicitly.** Copilot serves two payload dialects: camelCase event
|
||||
names give camelCase fields, and PascalCase names give the snake_case shape that VS Code and Claude
|
||||
use. `--event` means the mapping does not depend on which one a configuration chose.
|
||||
|
||||
### VS Code Chat
|
||||
|
||||
VS Code Chat has no hooks, so nothing observes a session while it runs. Its chronicle store
|
||||
(`session-store.db`, the same schema as Copilot CLI's) keeps sessions, turns and touched files, and
|
||||
[tools/import_chronicle.py](../../tools/import_chronicle.py) rebuilds a trace from it afterwards:
|
||||
|
||||
```bash
|
||||
tools/import_chronicle.py --dry-run # this repository's sessions
|
||||
tools/import_chronicle.py --session 0f1e2d3c
|
||||
```
|
||||
|
||||
The store starts empty, and `/chronicle reindex` fills it. That command also syncs session data to
|
||||
your GitHub account, so run it deliberately rather than from a script. A rebuilt trace marks itself
|
||||
`reconstructed: true`, and its `completeness` names `tool.post` but not `tool.pre`: the store records
|
||||
that a file was touched, not that a tool was about to run.
|
||||
|
||||
### Mistral Vibe
|
||||
|
||||
[.vibe/hooks.toml](../../.vibe/hooks.toml) declares the three hooks Vibe has, and
|
||||
[.vibe/config.toml](../../.vibe/config.toml) keeps the telemetry settings in the repository rather
|
||||
than in someone's shell profile. Both were checked against the installed CLI's own loader
|
||||
(`mistral-vibe 2.24.2`) rather than against its documentation. What that turned up:
|
||||
|
||||
- **A failing hook cannot block anything.** With `strict = false`, the default, a crash or a timeout
|
||||
is only a warning. Only a hook that opts into `strict` can deny a tool call - the mirror image of
|
||||
Copilot.
|
||||
- **`post_agent` carries no response text,** only the session context. That is why it maps to
|
||||
`turn.end` rather than `assistant.message`.
|
||||
- **`enable_telemetry` defaults to `true`.** Turning it off is a real change, and it also gates OTel
|
||||
export, which needs both flags on. This repository leaves both off: Vibe's `enable_otel` sends
|
||||
prompts to Mistral Studio.
|
||||
- **Vibe already reads `.agents/skills/` and `AGENTS.md`.** The directory that
|
||||
`wikitool instructions sync` publishes is a project-scope skill source for Vibe, so working with
|
||||
Vibe needs no adaptation, only a trusted folder.
|
||||
@@ -0,0 +1,88 @@
|
||||
# Troubleshoot an instance
|
||||
|
||||
What a common failure means, and what to do about it. During installation itself, the preflight
|
||||
names its own stops - what is missing, why, and the command that fixes it - and
|
||||
[INSTALL.md](../../INSTALL.md) explains them. This page covers an instance that is already set up.
|
||||
|
||||
For the MCP read server, see the troubleshooting section of
|
||||
[Set up the MCP read server](set-up-the-mcp-server.md#troubleshooting).
|
||||
|
||||
## `tools/wikitool` ends with `STOP - this checkout is not set up yet` (exit 42)
|
||||
|
||||
The preflight has not run in this checkout yet, or not since the last update. Run it:
|
||||
|
||||
```bash
|
||||
tools/preflight.sh
|
||||
```
|
||||
|
||||
In PowerShell 7 on Windows, the call is
|
||||
`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1`. The `-ExecutionPolicy Bypass`
|
||||
applies to that one process only and changes no setting.
|
||||
[instructions/preflight.md](../../instructions/preflight.md) is the procedure, and its output says
|
||||
what to install if anything is missing.
|
||||
|
||||
## The agent offers no skills (`wiki-ingest`, `wiki-query`, ...)
|
||||
|
||||
`.agents/skills/` and `.claude/skills/` are generated and not committed. Run:
|
||||
|
||||
```bash
|
||||
tools/wikitool instructions sync
|
||||
```
|
||||
|
||||
Then restart the agent session. A harness reads its skills only when the session starts.
|
||||
|
||||
## `doctor` reports `personalization: FAIL`
|
||||
|
||||
`USER.md` or `SOUL.md` is missing, or still carries the sentinel line from its template. A renamed
|
||||
template does not count as a filled one. Ask the agent to run the personalization step of
|
||||
[instructions/setup-instance.md](../../instructions/setup-instance.md). For a further clone of an
|
||||
older instance, that is the only step to catch up on.
|
||||
|
||||
## `doctor` reports `environment: WARN`
|
||||
|
||||
`ENVIRONMENT.md` exists but still carries the template's sentinel line. Fill it in (the template is
|
||||
`ENVIRONMENT.md.template`) and remove the line, or delete the file. It is optional, and `absent` is
|
||||
a valid state.
|
||||
|
||||
## `new` stops with "No author configured"
|
||||
|
||||
Neither `$WIKI_AUTHOR` nor `git config user.name` is set. Run `git config user.name "<Name>"`, or
|
||||
export `WIKI_AUTHOR` ([configuration](../reference/configuration.md)).
|
||||
|
||||
## `publish` ends with exit 42 (Mass-Update Gate)
|
||||
|
||||
Expected when ten or more files are counted, for example on the very first commit of a new instance
|
||||
or after a lint pass. It is not an error. It asks that a human see the output: the complete file
|
||||
list and the exact `--confirm <token>` line that publishes once you approve.
|
||||
[instructions/gates.md](../../instructions/gates.md) has the details.
|
||||
|
||||
## `publish` ends with exit 42 (Publish-Remote Gate)
|
||||
|
||||
This checkout has a `.wikitool-remotes.json`, and the remote being pushed to is not listed in it.
|
||||
Also not an error: the output names the push URL it would write to and the URLs that are allowed.
|
||||
Unlike the Mass-Update Gate, there is **no token and no flag** here. If the target really is right,
|
||||
you add its URL to the file yourself. An agent that edits the file to get past the refusal is
|
||||
opening a gate on its own initiative, which it must never do.
|
||||
|
||||
## Setup or an update fails, and you want to report it
|
||||
|
||||
Ask the agent to follow [instructions/bug-report.md](../../instructions/bug-report.md), or call
|
||||
`tools/bugreport` yourself. It works the same from Bash, Git Bash and PowerShell. It finds a
|
||||
Python 3.8 or newer by itself, skipping the Store aliases `python3` points at on Windows. It needs
|
||||
neither the preflight nor a working `wikitool`, and if it finds no Python, it says why.
|
||||
|
||||
The script writes a bundle to `reports/bugreport-<timestamp>/`, with a zip beside it.
|
||||
|
||||
- **What is removed:** secrets, and page titles from everything the script itself produces.
|
||||
`--titles` keeps the titles.
|
||||
- **What may still be in it:** the session trace is included by default (`--no-trace` leaves it
|
||||
out). Like the chronology and the transcripts, it can contain page content and titles. The bundle
|
||||
also contains machine, user and path names.
|
||||
- **Pseudonymising:** `--pseudonymise` replaces user, host, path, git and remote names with
|
||||
placeholders that keep their length, spaces, hyphens and path depth. The agent can then add more
|
||||
names - people, companies, customers, projects - with `--bundle … --candidates …`. That is a
|
||||
model's judgment, and it leaves a remainder: read the bundle before you share it. The mapping,
|
||||
the checklist and the candidates file contain the originals, and they sit beside the bundle,
|
||||
never in it.
|
||||
|
||||
Nothing is uploaded anywhere. You choose the channel.
|
||||
@@ -0,0 +1,145 @@
|
||||
# Update an instance
|
||||
|
||||
Bring an instance installed from a release up to a newer release of the stack: the tools, types,
|
||||
instructions and contracts. Your pages stay yours. The agent carries out the upgrade by following
|
||||
[instructions/upgrade-instance.md](../../instructions/upgrade-instance.md), which sets the order of
|
||||
the steps, what each one decides, and where the session has to restart. This page covers the
|
||||
decision before it, and what you'll see along the way.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [What is running, and is there something newer?](#what-is-running-and-is-there-something-newer)
|
||||
- [Read the notes first](#read-the-notes-first)
|
||||
- [What you say](#what-you-say)
|
||||
- [Where the agent stops](#where-the-agent-stops)
|
||||
- [Where the release comes from, and whom you trust](#where-the-release-comes-from-and-whom-you-trust)
|
||||
- [Two first-time cases](#two-first-time-cases)
|
||||
- [Pitfalls](#pitfalls)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## What is running, and is there something newer?
|
||||
|
||||
Every instance carries the version of the **stack**, not of its content. It is in `VERSION`, and
|
||||
`.wikitool-release.json` beside it records where the release came from and when it was exported.
|
||||
|
||||
```bash
|
||||
tools/wikitool version # what runs here, and where it came from
|
||||
tools/wikitool version check # is there a newer release?
|
||||
tools/wikitool version notes # what the newest release changes
|
||||
```
|
||||
|
||||
`version check`, `version notes` and `dist upgrade --latest` are the only commands that go to the
|
||||
network. All three ask the same release feed: the one named in the stamp, unless
|
||||
`WIKITOOL_UPDATE_URL` overrides it. A feed that cannot be reached is reported as an error, **never**
|
||||
as "up to date" and never as "no notes".
|
||||
|
||||
## Read the notes first
|
||||
|
||||
What a version number promises: two versions are compatible when they agree in the *leftmost
|
||||
non-zero part*. `0.1.3 → 0.1.4` is a safe update and `0.1.3 → 0.2.0` is not. From `1.0.0` on, the
|
||||
same rule reads as the familiar "a MAJOR breaks". `version check` says which case you are in
|
||||
(`state: update` or `state: migration`).
|
||||
|
||||
That number only answers whether the new machinery can simply be copied over the old. Whether your
|
||||
*content* has to be migrated is a second, independent question. So the release notes of a MAJOR
|
||||
carry two separate lines, and both are worth reading before you start:
|
||||
|
||||
- **Breaking Change:** what stops working, and what this instance has to do about it.
|
||||
- **Migration:** whether and how the pages are rewritten (`none required` when they are not).
|
||||
|
||||
`version notes` prints both. On an instance it reads them from the release feed, because an
|
||||
instance's own `CHANGES.md` is a stub that an update never overwrites. It always asks for the
|
||||
**newest** release, so while `VERSION` still names the old one, it answers with a version other than
|
||||
the one installed, and says so on stderr. If the feed cannot be reached, the error names the release
|
||||
page recorded as `release_url` in `.wikitool-release.json`. `--offline` sends you there directly.
|
||||
|
||||
Why the two questions are kept apart: [version-model.md](../explanation/version-model.md).
|
||||
|
||||
## What you say
|
||||
|
||||
```text
|
||||
Update this instance to the latest release
|
||||
```
|
||||
|
||||
## Where the agent stops
|
||||
|
||||
In the order you'll meet them:
|
||||
|
||||
- **The release notes.** The agent shows you both lines before it touches anything. A migration the
|
||||
instance already owes from an earlier release is finished first.
|
||||
- **A stack file you changed by hand.** `dist upgrade` lists every file that differs from what the
|
||||
installed release shipped, instead of overwriting it silently. For each one, you decide: take the
|
||||
release's version, or keep yours. Keeping yours means the file is reported again at every future
|
||||
upgrade.
|
||||
- **The preflight.** A new release may need other tools than the old one. If something is missing,
|
||||
the preflight stops with exit 42 and says what to install, exactly as during installation.
|
||||
- **The first publish.** The swapped machinery is published on its own, before any content changes.
|
||||
That is far more than ten files, so the Mass-Update Gate (exit 42) shows you the list and the
|
||||
`--confirm` line.
|
||||
- **The session restart.** The running session still holds the *old* `AGENTS.md`, contracts and
|
||||
skills, so the agent asks you to restart it before migrating content. The new session picks up
|
||||
where the old one stopped.
|
||||
- **An offered migration.** A required migration is simply carried out. An *offered* one changes a
|
||||
file the instance owns, so taking it or declining it is your decision.
|
||||
- **The second publish,** of the migrated content, behind the same gate.
|
||||
|
||||
If a step fails for a reason that isn't obvious, the agent stops and offers you a bug report rather
|
||||
than improvising ([Troubleshoot](troubleshoot.md)).
|
||||
|
||||
`doctor` warns as long as the content's version (`kb_version` in `.wikitool-kb.json`) is behind
|
||||
`VERSION` and migrations are still open. An instance older than `.wikitool-kb.json` has no such file
|
||||
at all. In that case `tools/wikitool migrate baseline <version>` is called once, and nothing is
|
||||
guessed.
|
||||
|
||||
## Where the release comes from, and whom you trust
|
||||
|
||||
`tools/wikitool dist upgrade --latest --expect <version>` asks the release feed for its newest
|
||||
release, downloads the tarball and its `.sha256` into a scratch directory, checks one against the
|
||||
other and applies the release. The scratch directory is removed on every exit, including
|
||||
`--dry-run`. `--expect` is the version `version notes` showed you: the feed only knows its *newest*
|
||||
release, and `--expect` refuses **before** downloading if a newer one has appeared since, rather
|
||||
than applying something nobody read. Pre-releases (`-beta.N`) need `--pre`.
|
||||
|
||||
**What the checksum does, and what it does not do.** The `.sha256` comes from the same feed as the
|
||||
tarball. It protects against a damaged transfer, not against a compromised feed. Whether a release
|
||||
is genuine rests on your trust in the host whose feed the instance asks (`update_url` in the stamp).
|
||||
Nothing forces https. A configured `http://` feed is a deliberate choice. `WIKITOOL_UPDATE_TOKEN`
|
||||
is sent only to downloads on the same host as the feed.
|
||||
|
||||
**Offline, or with a tarball from someone else:** put the tarball and its `.sha256` side by side,
|
||||
and the agent passes the file instead of `--latest` (`dist upgrade <tarball>`). A tarball has to
|
||||
unpack to exactly one top-level directory, which is the shape the release workflow builds.
|
||||
|
||||
## Two first-time cases
|
||||
|
||||
**The first jump to a release that knows `--latest`:** the option does not exist in the instance
|
||||
yet, because the instruction there belongs to the release being left. Fetch the tarball and the
|
||||
`.sha256` once from the release page, put them side by side, and run `dist upgrade <tarball>`. From
|
||||
the next release on, the instance has `--latest` itself.
|
||||
|
||||
**The first jump to `4.5.0` or later:** `dist upgrade` itself does not exist in the instance yet,
|
||||
because it arrived with `4.5.0`. Use the tool from the unpacked *new* tarball and point it at the old
|
||||
instance:
|
||||
|
||||
```bash
|
||||
tar -xzf chemenu-stack-<version>.tar.gz
|
||||
CHEMENU_ROOT="$PWD" chemenu-stack-<version>/tools/wikitool \
|
||||
dist upgrade chemenu-stack-<version>.tar.gz --dry-run
|
||||
```
|
||||
|
||||
`CHEMENU_ROOT` tells the package which corpus to work on ([configuration](../reference/configuration.md)).
|
||||
Without it, the package would take the unpacked tarball itself for the instance. From the second
|
||||
upgrade on, the instance carries the command and the instruction itself.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- An instance without a local `.wikitool-release.json`, or with one that has no `files` block (from
|
||||
before `4.5.0`), gives `dist upgrade` nothing to detect a local change against, so it refuses
|
||||
the swap. There is no repair for that today.
|
||||
- Given a tarball or directory, `dist upgrade` downloads nothing. Only `--latest` downloads.
|
||||
- To follow the swap by hand anyway, perhaps with a tool `wikitool` cannot run, use the file list in
|
||||
the `files` block of `.wikitool-release.json`. The exceptions - `kb/CONVENTIONS.md`, every
|
||||
`kb/*/COLLECTION.md`, `.wikitool-kb.json` - are in the
|
||||
[`dist upgrade` record](../reference/commands/distribution-and-versioning.md#dist-upgrade), along
|
||||
with what the command classifies and refuses.
|
||||
@@ -13,6 +13,9 @@ written by hand.
|
||||
start it, generated from their `SKILL.md`.
|
||||
- [Page types](page-types.md) - every page type the stack ships for `kb/`, its subtypes, where
|
||||
its pages are written and its required fields, generated from the type-specs.
|
||||
- [Configuration](configuration.md) - the environment variables `wikitool` reads and the
|
||||
per-checkout files an operator may create. Written by hand: it has no source to be generated
|
||||
from yet.
|
||||
|
||||
The last two describe the stack's shipped defaults, which an instance may change or extend;
|
||||
`tools/wikitool instructions list` and `tools/wikitool types describe <type>` show what one
|
||||
|
||||
@@ -55,7 +55,7 @@ tools/wikitool dist export ../my-wiki
|
||||
**Behaviour**
|
||||
|
||||
- Writes a contentless, distributable copy of this repo's machinery into an empty or new `<target>` directory.
|
||||
- Ships `AGENTS.md`/`README.md`/`EVALS.md` with any `dist:strip-start`...`dist:strip-end` marker region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs, their schemas and their subtype templates `types/<type>.<value>.md` re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim (minus `docs/dev/`), `tools/` (no venv/caches, and not the dev-only test suite `tools/chemenu/tests/` with its `pytest.ini`/`.coveragerc`), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas) and `VERSION`.
|
||||
- Ships `AGENTS.md`/`README.md`/`INSTALL.md` with any `dist:strip-start`...`dist:strip-end` marker region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs, their schemas and their subtype templates `types/<type>.<value>.md` re-keyed as `.template`, the stack's own verbatim), `docs/` verbatim (minus `docs/dev/`), `tools/` (no venv/caches, and not the dev-only test suite `tools/chemenu/tests/` with its `pytest.ini`/`.coveragerc`), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas) and `VERSION`.
|
||||
- Ships `raw/` and `incoming/` as flat roots, each with a `.gitkeep` and no subdirectories. `incoming/.gitkeep` is trackable and survives becoming a git repository, so a plain clone gets the directory without any bootstrap step.
|
||||
- Ships templates, never the filled files: `USER.md.template`/`SOUL.md.template`, `kb/CONVENTIONS.md.template`, and each collection's contract re-keyed as `kb/<name>/COLLECTION.md.template`. The filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb/<name>/COLLECTION.md`/`types/<page-type>.md`/`types/<page-type>.<value>.md` bind their instance; `find_leaks` refuses a plan carrying one.
|
||||
- Writes a generated `.wikitool-release.json` stamp: version, export date, origin, and a sha256 per exported file - the base a later upgrade compares against.
|
||||
@@ -242,7 +242,7 @@ tools/wikitool dist upgrade --latest --expect 8.0.0 --dry-run
|
||||
- `wikitool version check` - finds out whether an update exists
|
||||
- `wikitool version notes` - the notes of the release `--expect` should name
|
||||
- `instructions/upgrade-instance.md` - the order after the swap
|
||||
- `INSTALL.md` § "Version und Updates" - which release, whether to take it, where the tarball comes from
|
||||
- `docs/how-to/update-an-instance.md` - which release, whether to take it, where the tarball comes from
|
||||
- `wikitool migrate status` - the migrations the report names
|
||||
|
||||
## `version show`
|
||||
@@ -323,7 +323,7 @@ tools/wikitool version check --json
|
||||
- The only command whose whole job is the network call - `version notes` reaches the same feed too, but only as a fallback on a distributed instance, and `dist upgrade --latest` asks it which release to download.
|
||||
- Never reached implicitly from another command (`dist upgrade` asks only when passed `--latest`), needs no key, and times out after `--timeout` seconds (default 10).
|
||||
- The feed is `--url`, else `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin. `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously.
|
||||
- For `update` or `migration` it prints that applying the release is a separate, manual step (`INSTALL.md` § "Eine Instanz aktualisieren").
|
||||
- For `update` or `migration` it prints that applying the release is a separate, manual step (`docs/how-to/update-an-instance.md`).
|
||||
- Read-only; exempt from the Iteration Budget Gate.
|
||||
|
||||
**Properties**
|
||||
@@ -344,7 +344,7 @@ tools/wikitool version check --json
|
||||
|
||||
- `wikitool dist upgrade` - applies the update this reports
|
||||
- `wikitool version notes` - prints the release notes
|
||||
- `INSTALL.md` § "Version und Updates" - what the operator decides before an update
|
||||
- `docs/how-to/update-an-instance.md` - what the operator decides before an update
|
||||
|
||||
## `version notes`
|
||||
|
||||
|
||||
@@ -60,7 +60,7 @@ tools/wikitool doctor --json
|
||||
|
||||
- `instructions/setup-instance.md` - the setup steps most findings point back to
|
||||
- `instructions/preflight.md` - what `tool-paths` and `install-dir` point back to
|
||||
- `INSTALL.md` § "Konfiguration" - the per-checkout configuration files
|
||||
- `EVALS.md` - telemetry state and caps
|
||||
- `docs/reference/configuration.md` - the per-checkout configuration files
|
||||
- `docs/explanation/telemetry-and-evaluation.md` - telemetry state and caps
|
||||
- `instructions/session-setup.md` - setting `WIKITOOL_SESSION_ID`
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -401,7 +401,7 @@ tools/wikitool upload list --json
|
||||
**See also**
|
||||
|
||||
- `wikitool upload show` - one submission's manifest
|
||||
- `INSTALL-MCP.md` § "Schritt 7: Optional - den `submit`-Pfad freischalten"
|
||||
- `docs/how-to/set-up-the-mcp-server.md` § "Optional: the submit path"
|
||||
|
||||
## `upload show`
|
||||
|
||||
|
||||
@@ -53,7 +53,7 @@ tools/wikitool eval sessions --json
|
||||
**See also**
|
||||
|
||||
- `wikitool eval score` - scores one of them
|
||||
- `EVALS.md` - how telemetry and evaluation work
|
||||
- `docs/explanation/telemetry-and-evaluation.md` - how telemetry and evaluation work
|
||||
|
||||
## `eval score`
|
||||
|
||||
@@ -110,5 +110,5 @@ tools/wikitool eval score --session wiki-1727330000 --save
|
||||
**See also**
|
||||
|
||||
- `wikitool eval sessions` - which session ids exist
|
||||
- `EVALS.md` - the scoring levels
|
||||
- `docs/explanation/telemetry-and-evaluation.md` - the scoring levels
|
||||
<!-- /wikitool:reference-commands -->
|
||||
@@ -355,7 +355,7 @@ tools/wikitool docs toc --apply
|
||||
|
||||
- Creates, refreshes or removes the generated table-of-contents region on every reference file over 100 lines: `AGENTS.md`, every stage contract, `kb/CONVENTIONS.md`, every `kb/*/COLLECTION.md`, every flat `instructions/**.md` file, every `types/*.md` type-spec, and every `docs/` page - each together with the `<name>.template` it ships as, where one exists.
|
||||
- The scope is computed from those categories rather than listed, so a file added later is in scope without a code change.
|
||||
- Out of scope: every `SKILL.md`, the human docs (`README.md`, `CHANGES.md`, `EVALS.md`, `INSTALL.md`, `tools/README.md`), and every subtype template `types/<type>.<value>.md` - `new` copies it into a page, so it never carries a region whatever its length.
|
||||
- Out of scope: every `SKILL.md`, the human docs at the root and beside the code (`README.md`, `CHANGES.md`, `INSTALL.md`, `tools/README.md`), and every subtype template `types/<type>.<value>.md` - `new` copies it into a page, so it never carries a region whatever its length.
|
||||
- Dry-run by default (prints which files would change); `--apply` writes. Idempotent: a re-run after an interruption converges rather than doubling a region.
|
||||
- If `docs verify` still reports a stale region after `--apply`, the file's `##` headings changed in between; run it again.
|
||||
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
# Configuration
|
||||
|
||||
The environment variables `wikitool` reads, and the per-checkout files an operator may create at the
|
||||
root of an instance. None of them is needed for an instance to work: an instance installed from a
|
||||
release runs against the origin's public release feed with no configuration at all.
|
||||
|
||||
This page is written by hand. Unlike the command reference, it has no single source in the code it
|
||||
could be generated from yet, so a change to one of these files' shape has to be carried here
|
||||
by whoever makes it.
|
||||
|
||||
<!-- wikitool:toc -->
|
||||
## Contents
|
||||
|
||||
- [Environment variables](#environment-variables)
|
||||
- [Telemetry](#telemetry)
|
||||
- [Tool paths](#tool-paths)
|
||||
- [Allowed push URLs](#allowed-push-urls)
|
||||
- [Task tracker](#task-tracker)
|
||||
- [Super Productivity](#super-productivity)
|
||||
- [CalDAV](#caldav)
|
||||
- [MCP submissions](#mcp-submissions)
|
||||
<!-- /wikitool:toc -->
|
||||
|
||||
## Environment variables
|
||||
|
||||
| Variable | Purpose | Fallback |
|
||||
|----------|---------|----------|
|
||||
| `WIKI_AUTHOR` | Overrides the author name of new source pages | `git config user.name`. Without either, `new` stops with an `ERROR` |
|
||||
| `WIKITOOL_SESSION_ID` | Scopes the Iteration Budget Gate to a task rather than a terminal, and names the directory a trace lands in | A session variable the harness sets itself, where one is known (for example `CLAUDE_CODE_SESSION_ID`), else the parent process id ([instructions/session-setup.md](../../instructions/session-setup.md)) |
|
||||
| `WIKITOOL_UPDATE_URL` | The release feed `version check`, `version notes` and `dist upgrade --latest` ask | The value in `.wikitool-release.json`, else the origin's feed |
|
||||
| `WIKITOOL_UPDATE_TOKEN` | A Gitea token for the release feed, sent only to downloads on the feed's own host | None. Not needed against the origin, only against a feed in a private repository |
|
||||
| `WIKITOOL_TASKS_CONFIG` | Path to a tracker configuration that `task`, `review` and `doctor` read instead of `.wikitool-tasks.json`, for running one checkout against several trackers in turn | `.wikitool-tasks.json` at the root. A path to a file that does not exist is an error, never "no tracker configured" |
|
||||
| `CHEMENU_ROOT` | Which corpus the package works on, for a caller outside the checkout itself | The checkout the package sits in. `tools/wikitool` behaves the same without it |
|
||||
| `WIKI_TRACE` | `0`/`1` turns telemetry off or on, beating both the default and `.wikitool-telemetry.json` | The installation-form default ([below](#telemetry)) |
|
||||
| `WIKI_TRACE_DIR` | Writes traces somewhere other than `reports/telemetry/` | `reports/telemetry/` |
|
||||
| `WIKI_TRACE_CONTENT` | `0` keeps only lengths and digests of prompts and replies instead of their text | Text is kept |
|
||||
| `WIKI_TRACE_MAX_CONTENT` | Per-attribute content cap in characters | 60 KiB |
|
||||
| `WIKI_TRACE_MAX_SESSION_BYTES` | Byte cap per session trace; beats the file's `max_session_bytes` | 5 MiB |
|
||||
| `WIKI_TRACE_KEEP_SESSIONS` | How many session directories retention keeps; beats the file's `keep_sessions` | 250 |
|
||||
|
||||
## Telemetry
|
||||
|
||||
**Telemetry is off in an instance.** Every instance carries the `.wikitool-release.json` of its
|
||||
release, and that is how `chemenu.telemetry.policy` recognises that nobody asked for telemetry. The
|
||||
same holds for every further clone of the instance, because the file is committed with the first
|
||||
publish. Only a clone of the origin repository, used to develop the stack, has no stamp. There the
|
||||
traces are the instrument the stack measures itself with, and they are **on**.
|
||||
|
||||
To turn the default around, create `.wikitool-telemetry.json` at the root. It is per checkout and
|
||||
gitignored, with no `.template`, like `.wikitool-remotes.json`:
|
||||
|
||||
```json
|
||||
{ "enabled": true, "max_session_bytes": 5242880, "keep_sessions": 250 }
|
||||
```
|
||||
|
||||
All three keys are optional. `WIKI_TRACE` still overrides `enabled` in both directions and beats
|
||||
this file. `wikitool doctor` reports the current state: on or off, why, and how much has been
|
||||
recorded against both caps. What the caps do and what never reaches a trace:
|
||||
[telemetry-and-evaluation.md](../explanation/telemetry-and-evaluation.md).
|
||||
|
||||
## Tool paths
|
||||
|
||||
**The preflight writes this file, not you.** `.wikitool-tools.json` at the root holds the absolute
|
||||
paths of Python, git and ripgrep, as the preflight found them on this machine. `wikitool` starts git
|
||||
and rg from there instead of from `PATH`. It is per checkout and gitignored, because a path on one
|
||||
machine says nothing about the next.
|
||||
|
||||
If the file is missing or incomplete, `tools/wikitool` does not start (exit 42) and names the
|
||||
preflight. If a tool sits somewhere the preflight does not look, name it:
|
||||
`tools/preflight.sh --set rg=<path>`, or in PowerShell
|
||||
`pwsh -NoProfile -ExecutionPolicy Bypass -File tools/preflight.ps1 --set rg=<path>`. Never edit the
|
||||
file by hand. `doctor` reports under `tool-paths` whether every path still works.
|
||||
|
||||
## Allowed push URLs
|
||||
|
||||
`.wikitool-remotes.json` at the root restricts where `publish` may push, per checkout and
|
||||
gitignored:
|
||||
|
||||
```json
|
||||
{ "allowed_push_urls": ["git@example.org:me/my-wiki.git"] }
|
||||
```
|
||||
|
||||
Without the file, any remote is allowed. With it, `publish` refuses with exit 42 (the Publish-Remote
|
||||
Gate) before pushing to a URL not on the list. A malformed file is an error, not permission. Only you
|
||||
edit this file. The gate has no token and no flag, and an agent that edits the list to get past a
|
||||
refusal is opening a gate on its own initiative.
|
||||
|
||||
## Task tracker
|
||||
|
||||
**Optional.** The weekly review (`tools/wikitool review`, skill `gtd-weekly-review`) checks the
|
||||
project pages under `kb/gtd/` against a task tracker. Which tracker that is goes in
|
||||
`.wikitool-tasks.json` at the root. Like the telemetry and remotes files, it is per checkout and has
|
||||
no `.template`, and it is **gitignored as soon as a token is in it**. Without the file, no tracker is
|
||||
configured, which is a valid state, not an error. A broken file, on the other hand, is a `FAIL`: an
|
||||
unreadable configuration must never pass as "no tracker".
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": 1,
|
||||
"provider": "superproductivity",
|
||||
"thresholds": {
|
||||
"stalled_waiting_days": 14,
|
||||
"unpaged_project_weeks": 3,
|
||||
"someday_stale_months": 5
|
||||
},
|
||||
"superproductivity": {
|
||||
"access": "api",
|
||||
"api_base_url": "http://127.0.0.1:3876",
|
||||
"api_token": "<token from the SP settings>"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`provider` picks the adapter. The stack ships `superproductivity` and `caldav`.
|
||||
|
||||
The `thresholds` block holds the review's three thresholds (configuration, not schema): when a
|
||||
waiting-for counts as overdue, how old a tracker project without a `kb/` page has to be before it is
|
||||
reported, and when a someday item counts as stale. A value the provider cannot supply at all - a
|
||||
waiting item with no follow-up date, a tracker project with no creation date - is not skipped silently
|
||||
but reported as its own finding (`waiting_no_follow_up`/`project_age_unknown`).
|
||||
|
||||
The provider block of the same name holds the connection details.
|
||||
|
||||
### Super Productivity
|
||||
|
||||
`access` is **mandatory and has no fallback**. It chooses between two mutually exclusive ways in:
|
||||
|
||||
- **`access: "snapshot"`** for an instance run headless: it reads only the newest backup snapshot
|
||||
under `backups_dir`. That works without the app running, but only reads. The tracker cannot be
|
||||
written from there.
|
||||
|
||||
```json
|
||||
"superproductivity": {
|
||||
"access": "snapshot",
|
||||
"backups_dir": "~/.config/superProductivity/backups"
|
||||
}
|
||||
```
|
||||
|
||||
- **`access: "api"`** for a desktop instance: it talks only to the local REST API. That answers only
|
||||
while the app runs, but it gives the current state and carries the write path. `api_token` is
|
||||
required, because every endpoint except `GET /health` demands `Authorization: Bearer <token>`.
|
||||
|
||||
The block names only the fields of its own way in. A `backups_dir` beside `access: "api"`, or an
|
||||
`api_token` beside `access: "snapshot"`, is refused when the configuration is read, not ignored.
|
||||
|
||||
What can write:
|
||||
|
||||
- `tools/wikitool new project` creates a tracker project of the same name only with `access: "api"`,
|
||||
and even there not automatically. The command stops (exit 42) and asks a human to create it, then
|
||||
continues with `--resume`.
|
||||
- `tools/wikitool task new` creates a single item without a `kb/` page.
|
||||
- `tools/wikitool task close --id` marks an item done. It never deletes one.
|
||||
|
||||
With `access: "snapshot"`, all three refuse completely with exit 1, because the tracker is read-only
|
||||
from there. `tools/wikitool task list --project` only reads, so it answers with either `access`.
|
||||
|
||||
### CalDAV
|
||||
|
||||
`caldav` is the standards-based second adapter (RFC 4791/5545). It is verified against Nextcloud
|
||||
Tasks, with iOS *Reminders* as the mobile client, and built for that split: tick items off on the
|
||||
phone, maintain them at the desk. Unlike Super Productivity, there is only one way in. CalDAV is
|
||||
always a network access, so no `access` field is needed:
|
||||
|
||||
```json
|
||||
"caldav": {
|
||||
"url": "https://<host>/remote.php/dav/calendars/<user>/",
|
||||
"username": "<login>",
|
||||
"app_password": "<Nextcloud app password>",
|
||||
"inbox_list": "Inbox",
|
||||
"someday_list": "Someday",
|
||||
"exclude_lists": ["<an existing list that is not a project>"]
|
||||
}
|
||||
```
|
||||
|
||||
- `url` points at the account's CalDAV calendar home set. It may be an alias - Nextcloud accepts a
|
||||
short name there - because every later address comes only from the `href`s the server returns,
|
||||
never from this URL joined with a name.
|
||||
- `username` is the login name, which can differ from the user id in the URL. A Nextcloud app
|
||||
password is recommended, not the account password.
|
||||
- `inbox_list` and `someday_list` name the two fixed lists, exactly one of each per instance.
|
||||
- `exclude_lists` leaves out existing plain task lists that are not projects. A list with a
|
||||
`VEVENT` component never counts as a project anyway.
|
||||
|
||||
A project is a list whose only supported component is `VTODO`. `tools/wikitool new project` creates
|
||||
it automatically with `MKCALENDAR` - without asking, unlike Super Productivity, because CalDAV has a
|
||||
real create command for lists. The uniqueness check runs against **every** list in the account,
|
||||
including excluded lists, the inbox, someday and mixed calendars. If a new name collides with any of
|
||||
them, nothing is created and the collision is named. Renaming the existing list in Nextcloud stays a
|
||||
manual step.
|
||||
|
||||
`task new` and `task close` work without restriction on `caldav`, which has no read-only mode like
|
||||
`access: "snapshot"`. When ticking an item off, `task close` changes only `STATUS`, `COMPLETED`,
|
||||
`PERCENT-COMPLETE`, `LAST-MODIFIED` and `DTSTAMP` on the existing `.ics` resource. Every other
|
||||
property, including an unknown `X-` property or an alarm, is kept unchanged. A resource that changed
|
||||
since it was read (an ETag conflict) is not written, and the command stops with exit 1. Nothing is
|
||||
ever deleted. The lists of finished projects stay after archiving, because the adapter never deletes
|
||||
a list. Remove them by hand in Nextcloud when you want to.
|
||||
|
||||
## MCP submissions
|
||||
|
||||
`.wikitool-upload.json` in the served corpus turns on the MCP read server's optional `submit` tool.
|
||||
Its shape, and what the authentication in front of the server has to guarantee before you create
|
||||
it, are in [Set up the MCP read server](../how-to/set-up-the-mcp-server.md#optional-the-submit-path).
|
||||
Reference in new issue
Block a user