torbenandClaude Code
cceb9a7bfe
Files changed: - .gitignore - CHANGES.md - VERSION - docs/explanation/architecture.md - docs/explanation/pipeline-rationale.md - docs/how-to/README.md - docs/how-to/back-up-to-a-synced-folder.md - docs/how-to/collect-from-a-synced-folder.md - docs/how-to/ingest-a-source.md - docs/how-to/set-up-the-mcp-server.md - docs/reference/commands/README.md - docs/reference/commands/git.md - docs/reference/commands/instance-health.md - docs/reference/commands/raw-material-and-uploads.md - docs/reference/configuration.md - instructions/gates.md - instructions/ingest-queue.md - instructions/mcp-read-server.md - instructions/migrations/10.0.0-settings-tiers.md - instructions/wiki-ingest/SKILL.md - raw/CONTRACT.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/cli_contract.py - tools/chemenu/collect.py - tools/chemenu/commands/backup_cmd.py - tools/chemenu/commands/docs_verify.py - tools/chemenu/commands/doctor.py - tools/chemenu/commands/raw/pending.py - tools/chemenu/commands/upload_cmd.py - tools/chemenu/config.py - tools/chemenu/config_shapes.py - tools/chemenu/legacy_settings.py - tools/chemenu/mcp/server.py - tools/chemenu/reference_doc.py - tools/chemenu/tests/test_backup.py - tools/chemenu/tests/test_collect.py - tools/chemenu/tests/test_config_shapes.py - tools/chemenu/tests/test_doctor.py - tools/chemenu/tests/test_legacy_settings.py - tools/chemenu/tests/test_mcp_server.py - tools/chemenu/tests/test_upload.py - tools/chemenu/tests/test_upload_cmd.py - tools/chemenu/upload.py Co-Authored-By: Claude Code <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
283 lines
16 KiB
Markdown
283 lines
16 KiB
Markdown
# Architecture
|
|
|
|
What Chemenu is made of, and how the parts divide the work. For why the pipeline has exactly four
|
|
stages, see [pipeline-rationale.md](pipeline-rationale.md). This page is the map those reasons
|
|
apply to.
|
|
|
|
<!-- wikitool:toc -->
|
|
## Contents
|
|
|
|
- [A wiki that is compiled, not searched](#a-wiki-that-is-compiled-not-searched)
|
|
- [The language of the pages](#the-language-of-the-pages)
|
|
- [The tree](#the-tree)
|
|
- [Collections and areas](#collections-and-areas)
|
|
- [Contracts and READMEs](#contracts-and-readmes)
|
|
- [Who does what](#who-does-what)
|
|
- [The compiler: `tools/wikitool`](#the-compiler-toolswikitool)
|
|
- [The MCP read server](#the-mcp-read-server)
|
|
- [Built for IT work](#built-for-it-work)
|
|
- [Telemetry and evaluation](#telemetry-and-evaluation)
|
|
<!-- /wikitool:toc -->
|
|
|
|
## A wiki that is compiled, not searched
|
|
|
|
Chemenu is a pattern for a personal knowledge base maintained by an LLM. A plain retrieval setup goes
|
|
back to the raw documents for every question. Here, the LLM **builds and maintains a persistent
|
|
wiki** incrementally instead, and the wiki compounds over time. Knowledge is compiled once and kept
|
|
current, not re-derived for every question.
|
|
|
|
That one decision shapes everything else. Raw material is kept unchanged as evidence, and compiled
|
|
pages cite it. Every mechanical step of the compilation - frontmatter, cross-references, the catalog,
|
|
the log, publishing - is done by a deterministic tool, never by hand, so it comes out the same way
|
|
every time.
|
|
|
|
## The language of the pages
|
|
|
|
Source material in `raw/` is never touched and stays in whatever language it arrived in. The
|
|
compiled pages under `kb/` are written in the instance's own language. The origin repository's demo
|
|
corpus, for instance, is German. What stays in its original form inside a page is everything that
|
|
is an *identifier* rather than prose: page titles, section headings the tool owns, wikilink targets,
|
|
citation ids, schema enum values, tags, commands, paths and code. So `GitOps Ownership Model` and a
|
|
German section heading sit in the same page without contradiction.
|
|
|
|
Which lines count as identifiers is the stack's rule,
|
|
[kb/CONTRACT.md § Language and identifiers](../../kb/CONTRACT.md#language-and-identifiers).
|
|
*Which language* the prose is in, and what the tool-owned headings are called, is the instance's own
|
|
`kb/CONVENTIONS.md`. That is a per-instance decision, not a property of the pattern, so it lives in a
|
|
file the instance owns rather than in one the stack ships. A new instance picks its language when it
|
|
fills in `kb/CONVENTIONS.md`, before the first ingest. Why the control plane stays English
|
|
regardless: [language-boundaries.md](language-boundaries.md).
|
|
|
|
## The tree
|
|
|
|
```
|
|
chemenu/
|
|
├── AGENTS.md # Control plane: invariants, file naming, routing, gates
|
|
├── CLAUDE.md # Claude Code only: imports AGENTS.md, links the one Claude-Code-only decision (model/effort). No rules of its own
|
|
├── README.md # Landing page for humans
|
|
├── INSTALL.md # Human-readable install: one release, one sentence to the agent
|
|
├── CHANGES.md # Changelog for the stack itself
|
|
├── USER.md # Who operates this instance - context, never instructions
|
|
├── SOUL.md # How this instance sounds. AGENTS.md always wins over it
|
|
├── ENVIRONMENT.md # Optional, gitignored: this checkout's harness, MCP servers, remotes
|
|
├── *.md.template # Unfilled USER/SOUL/ENVIRONMENT - what a distribution ships instead
|
|
├── .gitignore # Anchored so nothing under raw/, kb/ or work/ is ever excluded;
|
|
│ # incoming/ is the one directory excluded the other way round
|
|
├── .github/hooks/ # Copilot CLI hooks - session tracing
|
|
├── .vibe/ # Mistral Vibe hooks + the repo's telemetry policy
|
|
├── docs/ # Human documentation
|
|
│ ├── tutorials/ # learning: a guided first walk
|
|
│ ├── how-to/ # doing: one task per page
|
|
│ ├── reference/ # looking up: commands, exit codes, skills, page types, configuration
|
|
│ └── explanation/ # understanding: why it is built this way (this page)
|
|
├── instructions/ # CONTROL: everything an agent is told to do
|
|
│ ├── CONTRACT.md # Instruction vs. skill, publishing, writing standard
|
|
│ ├── bootstrap.md # Prepare a further checkout of an instance
|
|
│ ├── gates.md # What to do when a gate refuses a call
|
|
│ ├── german-terminology.md # Which words stay English in German prose; register
|
|
│ ├── session-setup.md
|
|
│ ├── page-lifecycle.md
|
|
│ ├── publish-cycle.md
|
|
│ ├── ingest-large-tree.md
|
|
│ ├── ingest-queue.md # Reviewing a submission before wikitool upload accept promotes it
|
|
│ └── wiki-*/SKILL.md # Skills - copied into .agents/skills/ and .claude/skills/
|
|
├── quarantine/ # QUARANTINE (optional): where the intake channels (MCP `submit`, a synced
|
|
│ # folder, a command) put what nobody has reviewed, gitignored - read by
|
|
│ # no command in the ordinary pipeline; a human reviews it with
|
|
│ # `wikitool upload list/show/accept/reject`, `upload collect` fills it
|
|
├── incoming/ # INBOX: content gitignored - drop a file or one folder per source here, `raw accept` promotes it
|
|
├── raw/ # INPUT: immutable, untrusted source material
|
|
│ ├── CONTRACT.md # Date shard, capture fields, immutability, untrusted content
|
|
│ ├── 2026/09/ # Where `raw accept` puts a file: the month it was accepted
|
|
│ ├── articles/ # The old type directories: still valid paths, never moved,
|
|
│ ├── documents/ # but nothing new is ever routed into them again
|
|
│ ├── notes/
|
|
│ └── assets/
|
|
├── types/ # SCHEMA: the global type surface. Not a collection
|
|
│ ├── type-spec.md # Root contract: anatomy, placement, adding a type
|
|
│ ├── type-guidance.md # Contract for the *.guidance.md files below
|
|
│ ├── entity.md # Entity type config + template (+ .schema.yaml)
|
|
│ ├── entity.guidance.md # Its stack-owned authoring prose, shipped verbatim
|
|
│ ├── entity.person.md # Subtype template: what `new` scaffolds for entity_type=person
|
|
│ ├── concept.md # Concept type config + template (+ .guidance.md, + concept.decision.md)
|
|
│ ├── source.md # Source type config + template (+ .guidance.md)
|
|
│ ├── comparison.md # Comparison type config + template (+ .guidance.md)
|
|
│ ├── project.md # Project (Vorhaben) type config + template, no guidance file
|
|
│ ├── instruction.md # Instruction type - lives outside kb/ via `root: repo`
|
|
│ └── lint-report.md # Contract-only: describes reports/, owns no directory
|
|
├── kb/ # OUTPUT: compiled knowledge. A namespace, not a collection
|
|
│ ├── CONTRACT.md # Collections, naming, tone, linking, provenance
|
|
│ ├── index.md # Generated catalog *map*: counts and pointers
|
|
│ ├── log.md # Generated chronological audit log
|
|
│ ├── provenance.md # Generated raw-file reverse index
|
|
│ ├── entities/ # COLLECTION.md + INDEX.md + areas below
|
|
│ │ ├── codebases/
|
|
│ │ ├── systems/
|
|
│ │ ├── tools/ # own INDEX.md once past 50 pages
|
|
│ │ ├── technologies/
|
|
│ │ ├── people/
|
|
│ │ └── organizations/
|
|
│ ├── concepts/ # COLLECTION.md + INDEX.md + areas below
|
|
│ │ ├── architectures/
|
|
│ │ ├── patterns/
|
|
│ │ ├── protocols/
|
|
│ │ ├── workflows/
|
|
│ │ ├── decisions/
|
|
│ │ └── problems/
|
|
│ ├── sources/ # COLLECTION.md + INDEX.md + areas below
|
|
│ │ ├── transcripts/
|
|
│ │ ├── analyses/
|
|
│ │ ├── articles/
|
|
│ │ ├── documents/
|
|
│ │ ├── notes/
|
|
│ │ ├── trackers/
|
|
│ │ └── unclassified/
|
|
│ ├── comparisons/ # COLLECTION.md - comparison pages, no subtype axis
|
|
│ └── gtd/ # COLLECTION.md + INDEX.md + areas below
|
|
│ ├── haus/
|
|
│ ├── finanzen/
|
|
│ └── technik/
|
|
├── work/ # WORKSHOP: one directory per multi-session run, tracked
|
|
│ └── CONTRACT.md # Run keys, required files, how a run closes
|
|
├── reports/ # DERIVED: lint reports, traces, eval scores. Gitignored
|
|
│ └── CONTRACT.md
|
|
├── sites/ # The instance's own site handlers for `raw fetch`, one JSON file per website
|
|
│ └── CONTRACT.md # File format, which handler applies, what a broken file does
|
|
└── tools/ # COMPILER: the wikitool CLI
|
|
├── CONTRACT.md # Command records (generated) and index, maintenance schedule
|
|
└── README.md # How wikitool is built and how to change it
|
|
```
|
|
|
|
The collections and areas under `kb/` shown here are the origin repository's. An instance's own come
|
|
from its type-specs, as described below.
|
|
|
|
<!-- dist:strip-start -->
|
|
Only in the origin repository (`tools/CONTRACT.md` explains how an instance comes to lack them):
|
|
|
|
```
|
|
├── docs/dev/ # Human-readable: development environment and release workflow
|
|
└── commonplace/ # Vendored, read-only knowledge base
|
|
```
|
|
<!-- dist:strip-end -->
|
|
|
|
## Collections and areas
|
|
|
|
A directory under `kb/` is a **collection** exactly when it holds a `COLLECTION.md`. A subdirectory
|
|
inside one is an **area**, which inherits it, and that is as deep as a page goes. Nothing nests
|
|
below an area, because the generated catalog reads exactly two path segments under `kb/`, and it
|
|
would silently fold a deeper page into the area. `kb/CONTRACT.md` § Collections has the rule, and
|
|
`wikitool lint` reports a violation as a hard error.
|
|
|
|
Which areas a collection has is not chosen page by page. A type-spec's `layout:` maps its subtype
|
|
field onto directories, and `wikitool new` writes each page straight into the one its subtype names.
|
|
That is also what gives the catalog's shard threshold any effect: `index rebuild` splits **per
|
|
area**, so a collection with no areas keeps one table however large it grows. Once such a collection
|
|
is past the threshold, `wikitool lint` reports it as a recommendation rather than an error, together
|
|
with the split its subtype field would produce. It stays quiet when the split would not actually
|
|
help.
|
|
|
|
`kb/comparisons/` is the worked example of a collection that stays flat: it has no subtype field for
|
|
a `layout:` to key on at all. A *lopsided* subtype field is a different case, and it gets fixed
|
|
rather than left flat. `kb/sources/` looked lopsided only because `source_type` had a schema
|
|
`default:`, which the compiler applied whenever nobody chose a value. Once the default was removed
|
|
and the pages reclassified, it split into six real areas.
|
|
|
|
Every page type the stack ships, with its subtypes, its directories and its required fields, is in
|
|
[page-types.md](../reference/page-types.md). An instance adopts those type-specs and may change
|
|
them; `tools/wikitool types list` and `tools/wikitool types describe <type>` show what one instance
|
|
has.
|
|
|
|
## Contracts and READMEs
|
|
|
|
`COLLECTION.md` never appears outside `kb/`. The other layers carry a `CONTRACT.md` or a root
|
|
type-spec instead. A stage may carry both a `README.md` and a `CONTRACT.md`, because they have
|
|
different readers: the README is for humans working *on* that layer, and the contract binds an agent
|
|
working *with* it. `AGENTS.md` § File naming says which file is for which reader, and
|
|
`wikitool docs verify` checks it.
|
|
|
|
## Who does what
|
|
|
|
**You** curate sources by dropping them into `incoming/`, ask questions, review what changed in
|
|
`kb/log.md` and `kb/index.md`, and steer the agent on what to emphasise or look into. The
|
|
[how-to guides](../how-to/README.md) cover each of these.
|
|
|
|
**The agent** follows `AGENTS.md`, the cross-cutting schema and policy, and runs the step-by-step
|
|
procedures as skills. They are published into `.agents/skills/`, and mirrored into
|
|
`.claude/skills/` for Claude Code, by `tools/wikitool instructions sync`. Which skills ship, what
|
|
each does and which requests start it: [skills.md](../reference/skills.md).
|
|
|
|
**`tools/wikitool`** does every mechanical step a skill needs: scaffolding, renaming and deleting
|
|
pages, cross-references, regenerating the catalog, log and provenance index, structural linting,
|
|
decay math, and publishing. The agent calls it and never does that work by hand. Along the way it
|
|
maintains:
|
|
|
|
- `kb/index.md`, the catalog, always current;
|
|
- `kb/log.md`, the complete audit trail;
|
|
- `kb/provenance.md`, the reverse index from raw files to pages;
|
|
- pages of every type, each in the directory its type-spec names;
|
|
- lint reports, session traces and eval scores in `reports/`, which is gitignored.
|
|
|
|
## The compiler: `tools/wikitool`
|
|
|
|
Every command carries one data record kept next to its code: synopsis, properties, copyable
|
|
examples, each exit cause with what to do about it, prohibitions, and notes on its behaviour. The
|
|
same record is rendered four ways:
|
|
|
|
- `tools/wikitool <command> -h` prints it, and `tools/wikitool -h` prints a one-line index of every
|
|
command;
|
|
- a command that fails with exit 1 prints the record's reactions on stderr, right under its `ERROR`
|
|
line;
|
|
- [tools/CONTRACT.md](../../tools/CONTRACT.md) holds a generated copy for agents, together with the
|
|
maintenance schedule;
|
|
- [the command reference](../reference/commands/README.md) is a generated copy for humans.
|
|
|
|
`tools/wikitool docs verify` checks the copies, and every command's flags, against the CLI in both
|
|
directions. [tools/README.md](../../tools/README.md) is the other half: how the CLI is built and how
|
|
to add a command. `AGENTS.md` holds the invariants that say when each command is mandatory.
|
|
|
|
The wiki is also a git repository, and `tools/wikitool publish` is the one way its changes are
|
|
committed and pushed. That gives every change a version history, and the gates in front of
|
|
`publish` decide what needs a human to look first
|
|
([why-gates-are-code.md](why-gates-are-code.md)).
|
|
|
|
<!-- dist:strip-start -->
|
|
Extending `tools/wikitool`, the type schema or the instruction layer itself is a separate kind of
|
|
session, with rules of its own: the `stack-dev`, `stack-build` and `stack-close` skills under
|
|
`instructions/dev/`, which only the origin repository has.
|
|
<!-- dist:strip-end -->
|
|
|
|
## The MCP read server
|
|
|
|
The terminal is not the only way in. `tools/chemenu/mcp/` serves the same wiki read-only over MCP -
|
|
`search`, `types`, `describe_type`, `lint` and `status` - so a consumer that is not a shell on this
|
|
machine can ask the same questions and get the same answers. Literally the same: the CLI and the
|
|
server are two adapters over one core, and a golden test holds their output together rather than
|
|
trusting that it agrees.
|
|
|
|
There is no tool that writes to `kb/`, and not because one is filtered out of a list. The server
|
|
imports nothing under `chemenu/commands/`, so `new`, `publish` and the rest cannot be reached from
|
|
it. Every answer carries the commit it was computed from, so a checkout that has fallen behind gives
|
|
a visibly stale answer instead of a confident wrong one. The server's dependency is deliberately
|
|
not in `requirements.txt`: an instance that only uses the CLI should not have to install a web stack
|
|
for it.
|
|
|
|
Setting it up: [set-up-the-mcp-server.md](../how-to/set-up-the-mcp-server.md).
|
|
|
|
## Built for IT work
|
|
|
|
The shipped defaults are tailored to IT work:
|
|
|
|
- **entity types** for software development and systems - codebases, systems, tools, technologies;
|
|
- **relationship labels** such as "depends on", "uses" and "implements". The vocabulary is in the
|
|
instance's `kb/CONVENTIONS.md`, because it belongs to the instance rather than the stack;
|
|
- **templates** per page type, and per subtype where its pages need a shape of their own, such as a
|
|
person or a decision record (ADR);
|
|
- **guidelines** for documenting technical decisions, which can be exported back into the
|
|
repositories they are about ([export-guidelines.md](../how-to/export-guidelines.md));
|
|
- **cross-reference patterns** for code and architecture.
|
|
|
|
## Telemetry and evaluation
|
|
|
|
How the stack observes what an agent did, and how that record becomes a score, is its own page:
|
|
[telemetry-and-evaluation.md](telemetry-and-evaluation.md).
|