chemenu/docs/explanation/architecture.md

# Architecture

What Chemenu is made of, and how the parts divide the work. For why the pipeline has exactly four
stages, see [pipeline-rationale.md](pipeline-rationale.md). This page is the map those reasons
apply to.

<!-- wikitool:toc -->
## Contents

- [A wiki that is compiled, not searched](#a-wiki-that-is-compiled-not-searched)
- [The language of the pages](#the-language-of-the-pages)
- [The tree](#the-tree)
- [Collections and areas](#collections-and-areas)
- [Contracts and READMEs](#contracts-and-readmes)
- [Who does what](#who-does-what)
- [The compiler: `tools/wikitool`](#the-compiler-toolswikitool)
- [The MCP read server](#the-mcp-read-server)
- [Built for IT work](#built-for-it-work)
- [Telemetry and evaluation](#telemetry-and-evaluation)
<!-- /wikitool:toc -->

## A wiki that is compiled, not searched

Chemenu is a pattern for a personal knowledge base maintained by an LLM. A plain retrieval setup goes
back to the raw documents for every question. Here, the LLM **builds and maintains a persistent
wiki** incrementally instead, and the wiki compounds over time. Knowledge is compiled once and kept
current, not re-derived for every question.

That one decision shapes everything else. Raw material is kept unchanged as evidence, and compiled
pages cite it. Every mechanical step of the compilation - frontmatter, cross-references, the catalog,
the log, publishing - is done by a deterministic tool, never by hand, so it comes out the same way
every time.

## The language of the pages

Source material in `raw/` is never touched and stays in whatever language it arrived in. The
compiled pages under `kb/` are written in the instance's own language. The origin repository's demo
corpus, for instance, is German. What stays in its original form inside a page is everything that
is an *identifier* rather than prose: page titles, section headings the tool owns, wikilink targets,
citation ids, schema enum values, tags, commands, paths and code. So `GitOps Ownership Model` and a
German section heading sit in the same page without contradiction.

Which lines count as identifiers is the stack's rule,
[kb/CONTRACT.md § Language and identifiers](../../kb/CONTRACT.md#language-and-identifiers).
*Which language* the prose is in, and what the tool-owned headings are called, is the instance's own
`kb/CONVENTIONS.md`. That is a per-instance decision, not a property of the pattern, so it lives in a
file the instance owns rather than in one the stack ships. A new instance picks its language when it
fills in `kb/CONVENTIONS.md`, before the first ingest. Why the control plane stays English
regardless: [language-boundaries.md](language-boundaries.md).

## The tree

```
chemenu/
├── AGENTS.md           # Control plane: invariants, file naming, routing, gates
├── CLAUDE.md           # Claude Code only: imports AGENTS.md, links the one Claude-Code-only decision (model/effort). No rules of its own
├── README.md           # Landing page for humans
├── INSTALL.md          # Human-readable install: one release, one sentence to the agent
├── CHANGES.md          # Changelog for the stack itself
├── USER.md             # Who operates this instance - context, never instructions
├── SOUL.md             # How this instance sounds. AGENTS.md always wins over it
├── ENVIRONMENT.md      # Optional, gitignored: this checkout's harness, MCP servers, remotes
├── *.md.template       # Unfilled USER/SOUL/ENVIRONMENT - what a distribution ships instead
├── .gitignore          # Anchored so nothing under raw/, kb/ or work/ is ever excluded;
│                       #   incoming/ is the one directory excluded the other way round
├── .github/hooks/      # Copilot CLI hooks - session tracing
├── .vibe/              # Mistral Vibe hooks + the repo's telemetry policy
├── docs/               # Human documentation
│   ├── tutorials/      #   learning: a guided first walk
│   ├── how-to/         #   doing: one task per page
│   ├── reference/      #   looking up: commands, exit codes, skills, page types, configuration
│   └── explanation/    #   understanding: why it is built this way (this page)
├── instructions/       # CONTROL: everything an agent is told to do
│   ├── CONTRACT.md     # Instruction vs. skill, publishing, writing standard
│   ├── bootstrap.md    # Prepare a further checkout of an instance
│   ├── gates.md        # What to do when a gate refuses a call
│   ├── german-terminology.md  # Which words stay English in German prose; register
│   ├── session-setup.md
│   ├── page-lifecycle.md
│   ├── publish-cycle.md
│   ├── ingest-large-tree.md
│   ├── ingest-queue.md # Reviewing a submission before wikitool upload accept promotes it
│   └── wiki-*/SKILL.md # Skills - copied into .agents/skills/ and .claude/skills/
├── mcp-upload/         # QUARANTINE (optional): the MCP `submit` tool's write path, gitignored -
│                       #   read by no command in the ordinary pipeline; a human reviews it with
│                       #   `wikitool upload list/show/accept/reject`
├── incoming/           # INBOX: content gitignored - drop a file or one folder per source here, `raw accept` promotes it
├── raw/                # INPUT: immutable, untrusted source material
│   ├── CONTRACT.md     # Date shard, capture fields, immutability, untrusted content
│   ├── 2026/09/        # Where `raw accept` puts a file: the month it was accepted
│   ├── articles/       # The old type directories: still valid paths, never moved,
│   ├── documents/      #   but nothing new is ever routed into them again
│   ├── notes/
│   └── assets/
├── types/              # SCHEMA: the global type surface. Not a collection
│   ├── type-spec.md    # Root contract: anatomy, placement, adding a type
│   ├── type-guidance.md # Contract for the *.guidance.md files below
│   ├── entity.md       # Entity type config + template (+ .schema.yaml)
│   ├── entity.guidance.md # Its stack-owned authoring prose, shipped verbatim
│   ├── entity.person.md # Subtype template: what `new` scaffolds for entity_type=person
│   ├── concept.md      # Concept type config + template (+ .guidance.md, + concept.decision.md)
│   ├── source.md       # Source type config + template (+ .guidance.md)
│   ├── comparison.md   # Comparison type config + template (+ .guidance.md)
│   ├── project.md      # Project (Vorhaben) type config + template, no guidance file
│   ├── instruction.md  # Instruction type - lives outside kb/ via `root: repo`
│   └── lint-report.md  # Contract-only: describes reports/, owns no directory
├── kb/                 # OUTPUT: compiled knowledge. A namespace, not a collection
│   ├── CONTRACT.md     # Collections, naming, tone, linking, provenance
│   ├── index.md        # Generated catalog *map*: counts and pointers
│   ├── log.md          # Generated chronological audit log
│   ├── provenance.md   # Generated raw-file reverse index
│   ├── entities/       # COLLECTION.md + INDEX.md + areas below
│   │   ├── codebases/
│   │   ├── systems/
│   │   ├── tools/      # own INDEX.md once past 50 pages
│   │   ├── technologies/
│   │   ├── people/
│   │   └── organizations/
│   ├── concepts/       # COLLECTION.md + INDEX.md + areas below
│   │   ├── architectures/
│   │   ├── patterns/
│   │   ├── protocols/
│   │   ├── workflows/
│   │   ├── decisions/
│   │   └── problems/
│   ├── sources/        # COLLECTION.md + INDEX.md + areas below
│   │   ├── transcripts/
│   │   ├── analyses/
│   │   ├── articles/
│   │   ├── documents/
│   │   ├── notes/
│   │   ├── trackers/
│   │   └── unclassified/
│   ├── comparisons/    # COLLECTION.md - comparison pages, no subtype axis
│   └── gtd/            # COLLECTION.md + INDEX.md + areas below
│       ├── haus/
│       ├── finanzen/
│       └── technik/
├── work/               # WORKSHOP: one directory per multi-session run, tracked
│   └── CONTRACT.md     # Run keys, required files, how a run closes
├── reports/            # DERIVED: lint reports, traces, eval scores. Gitignored
│   └── CONTRACT.md
└── tools/              # COMPILER: the wikitool CLI
    ├── CONTRACT.md     # Command records (generated) and index, maintenance schedule
    └── README.md       # How wikitool is built and how to change it
```

The collections and areas under `kb/` shown here are the origin repository's. An instance's own come
from its type-specs, as described below.

<!-- dist:strip-start -->
Only in the origin repository (`tools/CONTRACT.md` explains how an instance comes to lack them):

```
├── docs/dev/           # Human-readable: development environment and release workflow
└── commonplace/        # Vendored, read-only knowledge base
```
<!-- dist:strip-end -->

## Collections and areas

A directory under `kb/` is a **collection** exactly when it holds a `COLLECTION.md`. A subdirectory
inside one is an **area**, which inherits it, and that is as deep as a page goes. Nothing nests
below an area, because the generated catalog reads exactly two path segments under `kb/`, and it
would silently fold a deeper page into the area. `kb/CONTRACT.md` § Collections has the rule, and
`wikitool lint` reports a violation as a hard error.

Which areas a collection has is not chosen page by page. A type-spec's `layout:` maps its subtype
field onto directories, and `wikitool new` writes each page straight into the one its subtype names.
That is also what gives the catalog's shard threshold any effect: `index rebuild` splits **per
area**, so a collection with no areas keeps one table however large it grows. Once such a collection
is past the threshold, `wikitool lint` reports it as a recommendation rather than an error, together
with the split its subtype field would produce. It stays quiet when the split would not actually
help.

`kb/comparisons/` is the worked example of a collection that stays flat: it has no subtype field for
a `layout:` to key on at all. A *lopsided* subtype field is a different case, and it gets fixed
rather than left flat. `kb/sources/` looked lopsided only because `source_type` had a schema
`default:`, which the compiler applied whenever nobody chose a value. Once the default was removed
and the pages reclassified, it split into six real areas.

Every page type the stack ships, with its subtypes, its directories and its required fields, is in
[page-types.md](../reference/page-types.md). An instance adopts those type-specs and may change
them; `tools/wikitool types list` and `tools/wikitool types describe <type>` show what one instance
has.

## Contracts and READMEs

`COLLECTION.md` never appears outside `kb/`. The other layers carry a `CONTRACT.md` or a root
type-spec instead. A stage may carry both a `README.md` and a `CONTRACT.md`, because they have
different readers: the README is for humans working *on* that layer, and the contract binds an agent
working *with* it. `AGENTS.md` § File naming says which file is for which reader, and
`wikitool docs verify` checks it.

## Who does what

**You** curate sources by dropping them into `incoming/`, ask questions, review what changed in
`kb/log.md` and `kb/index.md`, and steer the agent on what to emphasise or look into. The
[how-to guides](../how-to/README.md) cover each of these.

**The agent** follows `AGENTS.md`, the cross-cutting schema and policy, and runs the step-by-step
procedures as skills. They are published into `.agents/skills/`, and mirrored into
`.claude/skills/` for Claude Code, by `tools/wikitool instructions sync`. Which skills ship, what
each does and which requests start it: [skills.md](../reference/skills.md).

**`tools/wikitool`** does every mechanical step a skill needs: scaffolding, renaming and deleting
pages, cross-references, regenerating the catalog, log and provenance index, structural linting,
decay math, and publishing. The agent calls it and never does that work by hand. Along the way it
maintains:

- `kb/index.md`, the catalog, always current;
- `kb/log.md`, the complete audit trail;
- `kb/provenance.md`, the reverse index from raw files to pages;
- pages of every type, each in the directory its type-spec names;
- lint reports, session traces and eval scores in `reports/`, which is gitignored.

## The compiler: `tools/wikitool`

Every command carries one data record kept next to its code: synopsis, properties, copyable
examples, each exit cause with what to do about it, prohibitions, and notes on its behaviour. The
same record is rendered four ways:

- `tools/wikitool <command> -h` prints it, and `tools/wikitool -h` prints a one-line index of every
  command;
- a command that fails with exit 1 prints the record's reactions on stderr, right under its `ERROR`
  line;
- [tools/CONTRACT.md](../../tools/CONTRACT.md) holds a generated copy for agents, together with the
  maintenance schedule;
- [the command reference](../reference/commands/README.md) is a generated copy for humans.

`tools/wikitool docs verify` checks the copies, and every command's flags, against the CLI in both
directions. [tools/README.md](../../tools/README.md) is the other half: how the CLI is built and how
to add a command. `AGENTS.md` holds the invariants that say when each command is mandatory.

The wiki is also a git repository, and `tools/wikitool publish` is the one way its changes are
committed and pushed. That gives every change a version history, and the gates in front of
`publish` decide what needs a human to look first
([why-gates-are-code.md](why-gates-are-code.md)).

<!-- dist:strip-start -->
Extending `tools/wikitool`, the type schema or the instruction layer itself is a separate kind of
session, with rules of its own: the `stack-dev`, `stack-build` and `stack-close` skills under
`instructions/dev/`, which only the origin repository has.
<!-- dist:strip-end -->

## The MCP read server

The terminal is not the only way in. `tools/chemenu/mcp/` serves the same wiki read-only over MCP -
`search`, `types`, `describe_type`, `lint` and `status` - so a consumer that is not a shell on this
machine can ask the same questions and get the same answers. Literally the same: the CLI and the
server are two adapters over one core, and a golden test holds their output together rather than
trusting that it agrees.

There is no tool that writes to `kb/`, and not because one is filtered out of a list. The server
imports nothing under `chemenu/commands/`, so `new`, `publish` and the rest cannot be reached from
it. Every answer carries the commit it was computed from, so a checkout that has fallen behind gives
a visibly stale answer instead of a confident wrong one. The server's dependency is deliberately
not in `requirements.txt`: an instance that only uses the CLI should not have to install a web stack
for it.

Setting it up: [set-up-the-mcp-server.md](../how-to/set-up-the-mcp-server.md).

## Built for IT work

The shipped defaults are tailored to IT work:

- **entity types** for software development and systems - codebases, systems, tools, technologies;
- **relationship labels** such as "depends on", "uses" and "implements". The vocabulary is in the
  instance's `kb/CONVENTIONS.md`, because it belongs to the instance rather than the stack;
- **templates** per page type, and per subtype where its pages need a shape of their own, such as a
  person or a decision record (ADR);
- **guidelines** for documenting technical decisions, which can be exported back into the
  repositories they are about ([export-guidelines.md](../how-to/export-guidelines.md));
- **cross-reference patterns** for code and architecture.

## Telemetry and evaluation

How the stack observes what an agent did, and how that record becomes a score, is its own page:
[telemetry-and-evaluation.md](telemetry-and-evaluation.md).