Compare commits

...

17 Commits

Author SHA1 Message Date
torben 177c7e9ce8 feat: Prosa ist kein Identifier - Link-Taxonomie als Enum, generierte Regionen mit Markern (4.0.0)
CI / verify (push) Successful in 55s
Release / release (push) Successful in 38s
Files changed:
- .gitea/workflows/ci.yml
- AGENTS.md
- CHANGES.md
- VERSION
- instructions/CONTRACT.md
- instructions/link-taxonomy.md
- instructions/migrations/4.0.0-link-taxonomy.md
- instructions/setup-instance.md
- kb/CONTRACT.md
- kb/CONVENTIONS.md
- kb/CONVENTIONS.md.template
- kb/comparisons/COLLECTION.md
- kb/concepts/COLLECTION.md
- kb/entities/COLLECTION.md
- kb/sources/COLLECTION.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/blocks.py
- tools/chemenu/cli.py
- tools/chemenu/commands/cite_cmd.py
- tools/chemenu/commands/dist_cmd.py
- tools/chemenu/commands/docs_verify.py
- tools/chemenu/commands/doctor.py
- tools/chemenu/commands/links_cmd.py
- tools/chemenu/commands/migrate_cmd.py
- tools/chemenu/commands/new_page.py
- tools/chemenu/commands/page_ops.py
- tools/chemenu/commands/run_budget.py
- tools/chemenu/commands/xref.py
- tools/chemenu/conventions.py
- tools/chemenu/corpus_diff.py
- tools/chemenu/frontmatter_io.py
- tools/chemenu/kb_collections.py
- tools/chemenu/kb_state.py
- tools/chemenu/links.py
- tools/chemenu/lint_core.py
- tools/chemenu/provenance.py
- tools/chemenu/sections.py
- tools/chemenu/tests/conftest.py
- tools/chemenu/tests/test_blocks.py
- tools/chemenu/tests/test_cite_cmd.py
- tools/chemenu/tests/test_conventions.py
- tools/chemenu/tests/test_dist_cmd.py
- tools/chemenu/tests/test_doctor.py
- tools/chemenu/tests/test_migrate_cmd.py
- tools/chemenu/tests/test_new_page.py
- tools/chemenu/tests/test_pipeline_l0.py
- tools/chemenu/tests/test_types_cmd.py
- tools/chemenu/tests/test_xref.py
- types/concept.schema.yaml
- types/entity.md
- types/entity.schema.yaml
- types/instruction.schema.yaml
- types/type-spec.md
- work/link-taxonomy-migration/README.md
- work/link-taxonomy-migration/plan.md
2026-09-02 18:39:22 +02:00
torben 502971d147 feat: Autorenkonventionen nach Eigentum geschnitten - kb/CONVENTIONS.md, deklarierte Collections (3.0.0)
CI / verify (push) Successful in 53s
Release / release (push) Successful in 38s
Files changed:
- .gitea/workflows/ci.yml
- .wikitool-kb.json
- AGENTS.md
- CHANGES.md
- INSTALL.md
- README.md
- VERSION
- instructions/CONTRACT.md
- instructions/dev/testing-conventions.md
- instructions/german-terminology.md
- instructions/kb-profiles.md
- instructions/migrations/3.0.0-authoring-conventions.md
- instructions/private-instance.md
- instructions/setup-instance.md
- instructions/wiki-ingest/SKILL.md
- instructions/wiki-manage/SKILL.md
- kb/CONTRACT.md
- kb/CONVENTIONS.md
- kb/CONVENTIONS.md.template
- kb/comparisons/COLLECTION.md
- kb/concepts/COLLECTION.md
- kb/entities/COLLECTION.md
- kb/sources/COLLECTION.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/commands/dist_cmd.py
- tools/chemenu/commands/docs_verify.py
- tools/chemenu/commands/doctor.py
- tools/chemenu/commands/new_page.py
- tools/chemenu/conventions.py
- tools/chemenu/kb_collections.py
- tools/chemenu/kb_scan.py
- tools/chemenu/provenance.py
- tools/chemenu/sections.py
- tools/chemenu/tests/conftest.py
- tools/chemenu/tests/test_conventions.py
- tools/chemenu/tests/test_dist_cmd.py
- tools/chemenu/tests/test_doctor.py
- tools/chemenu/tests/test_new_page.py
- tools/chemenu/tests/test_types_cmd.py
- types/comparison.md
- types/concept.md
- types/entity.md
- types/source.md
- types/type-spec.md
2026-09-02 15:02:10 +02:00
torben 9843df99d3 ingest: raw/notes/Conversation Transcript - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02.md
Files changed:
- kb/concepts/INDEX.md
- kb/concepts/KB Stack Versioning.md
- kb/entities/INDEX.md
- kb/entities/projects/Chemenu.md
- kb/entities/tools/wikitool.md
- kb/index.md
- kb/log.md
- kb/sources/INDEX.md
- kb/sources/Source - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02.md
- raw/notes/Conversation Transcript - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02.md
2026-09-02 09:25:11 +02:00
torben 31662dc3ff feat: Versionsstelle als Kompatibilitaetsfrage, Breaking-Change-Vermerk erzwungen (2.5.0)
CI / verify (push) Successful in 54s
Release / release (push) Successful in 37s
Files changed:
- CHANGES.md
- INSTALL.md
- VERSION
- instructions/dev/stack-dev/SKILL.md
- instructions/dev/version-parts.md
- tools/CONTRACT.md
- tools/chemenu/commands/docs_verify.py
- tools/chemenu/commands/version_cmd.py
- tools/chemenu/tests/test_docs_verify.py
- tools/chemenu/tests/test_version_cmd.py
- tools/chemenu/version.py
2026-09-02 09:17:36 +02:00
torben 7fbb9a99df update: Korrektur zum stale Transkript-Verweis auf eine nie geschriebene zweite Sitzung
Files changed:
- kb/log.md
- kb/sources/Source - MCP Read Server Implementation Session 2026-09-02.md
2026-09-02 08:14:09 +02:00
torben 778764de4d update: qmd Sprachangabe korrigiert (TypeScript statt Go/Rust-Vermutung)
Files changed:
- kb/concepts/Delete Rather Than Anonymize.md
- kb/concepts/Dual Licensing by File Plan.md
- kb/concepts/Publish-Remote Gate.md
- kb/entities/INDEX.md
- kb/entities/tools/qmd.md
- kb/index.md
- kb/log.md
- kb/provenance.md
- kb/sources/INDEX.md
- kb/sources/Source - qmd - GitHub Repository.md
- raw/documents/qmd - GitHub Repository.md
2026-09-02 07:55:45 +02:00
torben f7597b209c ingest: MCP Read Server Implementation Session 2026-09-02
Files changed:
- kb/concepts/INDEX.md
- kb/concepts/Iteration and Cost Limits.md
- kb/concepts/MCP-Leseserver.md
- kb/concepts/Mass-Update Gate.md
- kb/concepts/Publish-Remote Gate.md
- kb/entities/INDEX.md
- kb/entities/projects/Chemenu.md
- kb/entities/tools/wikitool.md
- kb/index.md
- kb/log.md
- kb/provenance.md
- kb/sources/INDEX.md
- kb/sources/Source - MCP Read Server Implementation Session 2026-09-02.md
- raw/notes/Conversation Transcript - MCP Read Server Implementation Session 2026-09-02.md
2026-09-02 07:51:52 +02:00
torben 83018fcc7d docs: INSTALL-MCP.md fuer Menschen, wird mit dist export ausgeliefert (2.4.1)
CI / verify (push) Successful in 53s
Release / release (push) Successful in 38s
Files changed:
- CHANGES.md
- INSTALL-MCP.md
- INSTALL.md
- README.md
- VERSION
- tools/chemenu/commands/dist_cmd.py
2026-09-02 07:27:57 +02:00
torben 576df2cddd feat: MCP-Leseserver, Bibliotheksgrenze, Haertung des Lesepfads, Publish-Remote-Gate scharf (2.4.0)
CI / verify (push) Successful in 52s
Release / release (push) Successful in 37s
Files changed:
- .gitea/workflows/ci.yml
- CHANGES.md
- README.md
- VERSION
- instructions/mcp-read-server.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/api.py
- tools/chemenu/commands/doctor.py
- tools/chemenu/commands/lint.py
- tools/chemenu/commands/search.py
- tools/chemenu/commands/types_cmd.py
- tools/chemenu/config.py
- tools/chemenu/corpus_cache.py
- tools/chemenu/errors.py
- tools/chemenu/frontmatter_io.py
- tools/chemenu/lint_core.py
- tools/chemenu/mcp/__init__.py
- tools/chemenu/mcp/__main__.py
- tools/chemenu/mcp/server.py
- tools/chemenu/page.py
- tools/chemenu/search/filters.py
- tools/chemenu/search/registry.py
- tools/chemenu/search/ripgrep.py
- tools/chemenu/search/service.py
- tools/chemenu/tests/conftest.py
- tools/chemenu/tests/test_api.py
- tools/chemenu/tests/test_corpus_cache.py
- tools/chemenu/tests/test_doctor.py
- tools/chemenu/tests/test_frontmatter_io.py
- tools/chemenu/tests/test_instructions_cmd.py
- tools/chemenu/tests/test_mcp_server.py
- tools/chemenu/tests/test_new_page.py
- tools/chemenu/tests/test_search.py
- tools/chemenu/type_resolver.py
- tools/chemenu/types_core.py
- tools/requirements-mcp.txt
2026-09-02 07:19:32 +02:00
torben d1cf2e0327 ingest: Private-Instance Merge Correction and Issue 30 Session 2026-09-01
Files changed:
- kb/concepts/Delete Rather Than Anonymize.md
- kb/concepts/Publish-Remote Gate.md
- kb/entities/projects/Chemenu.md
- kb/entities/tools/wikitool.md
- kb/index.md
- kb/log.md
- kb/provenance.md
- kb/sources/INDEX.md
- kb/sources/Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01.md
2026-09-01 20:36:01 +02:00
torben df7ea93060 ingest: Publish-Remote Gate and Issue Triage Session 2026-09-01
Files changed:
- kb/concepts/INDEX.md
- kb/concepts/Mass-Update Gate.md
- kb/concepts/Publish-Remote Gate.md
- kb/entities/projects/Chemenu.md
- kb/entities/tools/wikitool.md
- kb/index.md
- kb/log.md
- kb/provenance.md
- kb/sources/INDEX.md
- kb/sources/Source - Publish-Remote Gate and Issue Triage Session 2026-09-01.md
2026-09-01 20:34:21 +02:00
torben 00c2cf6ffe ingest: Public Release, Corpus Purge and History Squash Session 2026-09-01
Files changed:
- kb/concepts/Delete Rather Than Anonymize.md
- kb/concepts/Dual Licensing by File Plan.md
- kb/concepts/INDEX.md
- kb/entities/projects/Chemenu.md
- kb/entities/tools/wikitool.md
- kb/index.md
- kb/log.md
- kb/provenance.md
- kb/sources/INDEX.md
- kb/sources/Source - Public Release, Corpus Purge and History Squash Session 2026-09-01.md
- raw/notes/Conversation Transcript - Private-Instance Merge Correction and Issue 30 Session 2026-09-01.md
- raw/notes/Conversation Transcript - Public Release, Corpus Purge and History Squash Session 2026-09-01.md
- raw/notes/Conversation Transcript - Publish-Remote Gate and Issue Triage Session 2026-09-01.md
2026-09-01 20:32:05 +02:00
torben 32a9b8eb3f docs: Publish-Remote Gate im Werkzeugvertrag, Projektseite auf oeffentlich (2.2.2)
CI / verify (push) Successful in 45s
Release / release (push) Successful in 37s
Files changed:
- CHANGES.md
- VERSION
- instructions/gates.md
- kb/entities/INDEX.md
- kb/entities/projects/Chemenu.md
- kb/log.md
- tools/CONTRACT.md
2026-09-01 19:15:36 +02:00
torben b2f7dec122 fix: private-instance - Demo-Korpus wandert beim Merge mit, Prozedur korrigiert (2.2.1)
CI / verify (push) Successful in 43s
Release / release (push) Successful in 38s
Files changed:
- CHANGES.md
- VERSION
- instructions/private-instance.md
2026-09-01 19:03:23 +02:00
torben 29063f511b docs: INSTALL.md auf das oeffentliche Repo umgestellt, Weg D fuer die private Instanz
CI / verify (push) Successful in 45s
Files changed:
- INSTALL.md
2026-09-01 18:14:08 +02:00
torben 7263f85936 feat: Publish-Remote Gate und die Anleitung fuer eine private Instanz (2.2.0)
CI / verify (push) Successful in 44s
Release / release (push) Successful in 36s
Files changed:
- .gitignore
- AGENTS.md
- CHANGES.md
- VERSION
- instructions/gates.md
- instructions/private-instance.md
- tools/chemenu/commands/doctor.py
- tools/chemenu/commands/git_publish.py
- tools/chemenu/config.py
- tools/chemenu/tests/test_git_publish.py
2026-09-01 18:06:55 +02:00
torben fb97d46888 docs: Formulierung im 2.1.0-Eintrag geglaettet
CI / verify (push) Successful in 43s
Files changed:
- CHANGES.md
2026-09-01 17:07:18 +02:00
134 changed files with 10730 additions and 1320 deletions
+14
View File
@@ -101,6 +101,10 @@ jobs:
# instance does not measure this suite. Installed beside pytest for # instance does not measure this suite. Installed beside pytest for
# the same reason pytest itself is. # the same reason pytest itself is.
tools/.venv/bin/pip install --quiet pytest pytest-cov tools/.venv/bin/pip install --quiet pytest pytest-cov
# The MCP server's dependency is optional for an instance but not for
# CI: its tests skip without it, and a skipped golden test is exactly
# how the server's output and the CLI's would drift apart unnoticed.
tools/.venv/bin/pip install --quiet -r tools/requirements-mcp.txt
- name: Tests - name: Tests
# Not run with WIKI_TRACE=0: two telemetry tests assert that a trace is # Not run with WIKI_TRACE=0: two telemetry tests assert that a trace is
@@ -217,6 +221,16 @@ jobs:
for personal in USER SOUL; do for personal in USER SOUL; do
grep -v 'wikitool:template-unfilled' "$personal.md.template" > "$personal.md" grep -v 'wikitool:template-unfilled' "$personal.md.template" > "$personal.md"
done done
# The authoring conventions ride the same split one directory down,
# and are stubbed the same way: what is under test is that the export
# carries the templates and that `doctor`/`docs verify` accept an
# adopted one, not what a person would write into them. The collection
# contracts are adopted verbatim - the shipped text is a working
# default, unlike a personalization file.
grep -v 'wikitool:template-unfilled' kb/CONVENTIONS.md.template > kb/CONVENTIONS.md
for template in kb/*/COLLECTION.md.template types/*.template; do
cp "$template" "${template%.template}"
done
python3 -m venv tools/.venv python3 -m venv tools/.venv
tools/.venv/bin/pip install --quiet -r tools/requirements.txt tools/.venv/bin/pip install --quiet -r tools/requirements.txt
tools/wikitool instructions sync tools/wikitool instructions sync
+7
View File
@@ -110,6 +110,13 @@ npm-debug.log*
# `dist export`; this anchored pattern deliberately does not match it. # `dist export`; this anchored pattern deliberately does not match it.
/ENVIRONMENT.md /ENVIRONMENT.md
# Publish-Remote Gate allowlist (see instructions/gates.md). Names the push
# URLs *this* checkout may publish to, so it is per-checkout for exactly the
# reason ENVIRONMENT.md above is: a committed copy would tell a private clone
# that the public upstream is a legitimate target for its own content. Absent
# means unrestricted; `doctor` reports which.
/.wikitool-remotes.json
# Coverage output from `pytest --cov` (see .gitea/workflows/ci.yml). Derived, # Coverage output from `pytest --cov` (see .gitea/workflows/ci.yml). Derived,
# like reports/: recomputable from any commit, and `publish` runs `git add -A`, # like reports/: recomputable from any commit, and `publish` runs `git add -A`,
# so an unignored htmlcov/ would commit itself on the next content publish. # so an unignored htmlcov/ would commit itself on the next content publish.
+8 -2
View File
@@ -1,5 +1,11 @@
{ {
"schema": 1, "schema": 1,
"kb_version": "1.0.0", "kb_version": "3.0.0",
"applied": [] "applied": [
{
"migration": "3.0.0-authoring-conventions",
"at": "2026-09-02",
"pages": 0
}
]
} }
+26 -8
View File
@@ -20,7 +20,9 @@ tools/wikitool instructions sync
Full procedure, including the tool environment: [instructions/bootstrap.md](instructions/bootstrap.md). Full procedure, including the tool environment: [instructions/bootstrap.md](instructions/bootstrap.md).
Setting up a brand-new, empty instance instead of cloning this one: `tools/wikitool dist export` Setting up a brand-new, empty instance instead of cloning this one: `tools/wikitool dist export`
and [instructions/setup-instance.md](instructions/setup-instance.md) - see and [instructions/setup-instance.md](instructions/setup-instance.md) - see
[INSTALL.md](INSTALL.md). [INSTALL.md](INSTALL.md). A *private* instance that keeps taking stack updates from a public
upstream is a third shape, with a safeguard the other two do not need:
[instructions/private-instance.md](instructions/private-instance.md).
## Invariants ## Invariants
@@ -75,10 +77,11 @@ What a file is called says who it is for and how it is loaded. This is a rule, n
| `SOUL.md` | Agents | Always, every session | | `SOUL.md` | Agents | Always, every session |
| `ENVIRONMENT.md` | Agents | Every session, **if it exists** - the one optional file in this table. Not committed: it describes one checkout, not the repo | | `ENVIRONMENT.md` | Agents | Every session, **if it exists** - the one optional file in this table. Not committed: it describes one checkout, not the repo |
| `<stage>/CONTRACT.md` | Agents | When writing in that stage | | `<stage>/CONTRACT.md` | Agents | When writing in that stage |
| `kb/<collection>/COLLECTION.md` | Agents | When writing in that collection | | `kb/CONVENTIONS.md` | Agents | When writing any page - it holds what *this* instance decided about authoring (language, section headings, naming, tone, relationship labels, confidence rubric), where `kb/CONTRACT.md` holds what the stack enforces. Instance-owned: a distribution ships only the `.template` |
| `kb/<collection>/COLLECTION.md` | Agents | When writing in that collection. Instance-owned in the same way, and declares in frontmatter which profile it adopted |
| `instructions/<name>.md` | Agents | By link, or on explicit request | | `instructions/<name>.md` | Agents | By link, or on explicit request |
| `instructions/<name>/SKILL.md` | Agents | By the harness, once published | | `instructions/<name>/SKILL.md` | Agents | By the harness, once published |
| `types/<name>.md` | Agents + validator | Via `tools/wikitool types describe` | | `types/<name>.md` | Agents + validator | Via `tools/wikitool types describe`. Split by `root:`: a page type-spec (`root: kb`) belongs to the instance and ships as `.template`; one describing a stack artifact ships verbatim |
| `INDEX.md` | Both | Generated - never hand-edited | | `INDEX.md` | Both | Generated - never hand-edited |
A stage may carry both a `README.md` and a `CONTRACT.md`: different readers, different A stage may carry both a `README.md` and a `CONTRACT.md`: different readers, different
@@ -102,6 +105,16 @@ and `SOUL.md.template`; the Personalization step of
writes the real files. `tools/wikitool doctor` FAILs on a missing one, and on one still writes the real files. `tools/wikitool doctor` FAILs on a missing one, and on one still
carrying the template's sentinel. carrying the template's sentinel.
The same `.template` split runs one directory down, for authoring rather than for voice.
`kb/CONVENTIONS.md` and each `kb/<name>/COLLECTION.md` bind every page and belong to the
instance, so a distribution ships them as templates and the KB-language step of
[instructions/setup-instance.md](instructions/setup-instance.md) fills them in, out of a
catalogue of ready-made profiles it routes to; `doctor` FAILs on a missing or unfilled
`kb/CONVENTIONS.md` the same way.
Unlike `USER.md`, these two *are* a source of rules: they are as binding as `kb/CONTRACT.md`.
What differs is ownership, not authority.
## Environment ## Environment
`ENVIRONMENT.md` records what *this checkout* works through - harness, published skills, `ENVIRONMENT.md` records what *this checkout* works through - harness, published skills,
@@ -140,15 +153,17 @@ Alongside it, not part of it: `instructions/` (what agents are told to do) and t
|-------|----------|--------| |-------|----------|--------|
| `raw/` | [raw/CONTRACT.md](raw/CONTRACT.md) | Immutability, directory routing, untrusted-content rule | | `raw/` | [raw/CONTRACT.md](raw/CONTRACT.md) | Immutability, directory routing, untrusted-content rule |
| `types/` | [types/type-spec.md](types/type-spec.md) | Type-spec anatomy, placement, adding a type, template variables | | `types/` | [types/type-spec.md](types/type-spec.md) | Type-spec anatomy, placement, adding a type, template variables |
| `kb/` | [kb/CONTRACT.md](kb/CONTRACT.md) | Collections, naming, tone, linking, provenance, confidence | | `kb/` | [kb/CONTRACT.md](kb/CONTRACT.md) + `kb/CONVENTIONS.md` | What the stack enforces about a page (collections, linking, provenance, confidence machinery), and beside it what this instance decided (language, naming, tone, labels, rubric) |
| `reports/` | [reports/CONTRACT.md](reports/CONTRACT.md) | Why reports and traces are generated, gitignored, and carried into `kb/log.md` | | `reports/` | [reports/CONTRACT.md](reports/CONTRACT.md) | Why reports and traces are generated, gitignored, and carried into `kb/log.md` |
| `work/` | [work/CONTRACT.md](work/CONTRACT.md) | Workshop runs: run keys, required files, why they are tracked, how a run closes | | `work/` | [work/CONTRACT.md](work/CONTRACT.md) | Workshop runs: run keys, required files, why they are tracked, how a run closes |
| `tools/` | [tools/CONTRACT.md](tools/CONTRACT.md) | Full command reference, per-command error contracts, maintenance schedule | | `tools/` | [tools/CONTRACT.md](tools/CONTRACT.md) | Full command reference, per-command error contracts, maintenance schedule |
| `instructions/` | [instructions/CONTRACT.md](instructions/CONTRACT.md) | Instruction vs. skill, publishing, writing standard | | `instructions/` | [instructions/CONTRACT.md](instructions/CONTRACT.md) | Instruction vs. skill, publishing, writing standard |
**By collection** - then read the contract for the collection you are writing in. **By collection** - then read the contract for the collection you are writing in.
[kb/CONTRACT.md](kb/CONTRACT.md) routes between `kb/entities/`, `kb/concepts/`, `kb/sources/` [kb/CONTRACT.md](kb/CONTRACT.md) routes between this instance's collections and holds the rules
and `kb/comparisons/`, and holds the rules they share. the stack enforces across all of them; `kb/CONVENTIONS.md` holds the ones this instance chose.
Both bind. The difference is who may change the sentence - which is also why a distribution
ships the first verbatim and the second only as a `.template`.
**By task** - skills hold the step-by-step procedures. Sources live in `instructions/<name>/`: **By task** - skills hold the step-by-step procedures. Sources live in `instructions/<name>/`:
@@ -177,16 +192,19 @@ tools/wikitool search --field entity_type=system --field 'confidence<0.6'
## Gates ## Gates
Two limits are enforced in code rather than by instruction, because a prompt-level limit is Three limits are enforced in code rather than by instruction, because a prompt-level limit is
one an agent can talk itself past. one an agent can talk itself past.
- **Mass-Update Gate.** `publish` exits **42** on a change touching too many files, printing - **Mass-Update Gate.** `publish` exits **42** on a change touching too many files, printing
the file list and the `--confirm <token>` line that publishes it once the user approves. The the file list and the `--confirm <token>` line that publishes it once the user approves. The
threshold and the rule live in [instructions/gates.md](instructions/gates.md). threshold and the rule live in [instructions/gates.md](instructions/gates.md).
- **Publish-Remote Gate.** `publish` exits **42** on a push to a URL this checkout has not
declared in `.wikitool-remotes.json`. It has no token and no flag: the way past it is a
deliberate edit by the user, never by an agent.
- **Iteration Budget Gate / Loop-Breaker.** Past 60 `wikitool` calls in a session, or after 3 - **Iteration Budget Gate / Loop-Breaker.** Past 60 `wikitool` calls in a session, or after 3
identical calls in a row, further calls are refused. identical calls in a row, further calls are refused.
Both refuse with exit 1. **Do not retry, and do not open the gate.** Stop, summarize the The last refuses with exit 1. **Do not retry, and do not open a gate.** Stop, summarize the
situation to the user, and get explicit approval. The full procedure - including why situation to the user, and get explicit approval. The full procedure - including why
`budget reset` is not the escape hatch - is [instructions/gates.md](instructions/gates.md). `budget reset` is not the escape hatch - is [instructions/gates.md](instructions/gates.md).
+613 -3
View File
@@ -20,6 +20,617 @@ their date-only headings.
--- ---
## 4.0.0 - 2026-09-02 - Prosa ist kein Identifier: Link-Taxonomie als Enum, generierte Regionen mit Markern
**Author:** Torben Nehmer
**Breaking Change:** Beziehungslabel sind Enum-Werte in related: statt Freitext im Body-Bullet, toolgefuehrte Abschnitte liegen zwischen Marker-Paaren statt hinter ihrer Ueberschrift, und xref add schreibt nur noch eine Kante statt beider Richtungen. tools/chemenu/sections.py ist geloescht. Eine bestehende Instanz muss sections: in kb/CONVENTIONS.md auf links/footnotes umstellen, outbound: in jede COLLECTION.md eintragen, die {section.*}-Variablen aus ihren Page-Type-Templates entfernen und den Korpus umstellen - sonst scaffoldet new die Variablen woertlich in neue Seiten. Ablauf: instructions/migrations/4.0.0-link-taxonomy.md
Der Stack benutzte an drei Stellen **Prosa als Identifier**, und jede hat messbar etwas
gekostet. Die Überschrift eines Abschnitts war seine Adresse (`^## Beziehungen$`), was die
KB-Sprache zu einer Compiler-Konstante machte *und* das Ende der Region zur Schätzung - sie lief
bis zur nächsten Überschrift, davor bis zum Dateiende, und hat auf acht Seiten still Inhalt
gelöscht. Das Beziehungslabel stand nur im Body-Bullet, also konnte nichts das Vokabular prüfen:
gemessen am Korpus **152 distinkte Label in 337 Bullets** gegen dreizehn dokumentierte, 102 davon
genau einmal vorkommend. Und `xref add` spiegelte jede Kante, was `## Siehe auch` mit 555
Bullets ohne Label füllte - 353 davon beweisbar redundant.
**Was jetzt Identifier ist.** Eine Region liegt zwischen `<!-- wikitool:links -->` bzw.
`<!-- wikitool:footnotes -->` und wird vollständig aus dem Frontmatter gerendert, Überschrift
eingeschlossen. Ein Label ist ein Maschinenwert in `related:` (`- depends-on: Hermes`), gezogen
aus `instructions/link-taxonomy.md` und **pro Ziel autorisiert von der Quell-Collection**
(`outbound:` im `COLLECTION.md`, Commonplaces ADR-019). Der Body-Bullet ist eine Darstellung
dieser Daten, nicht ihr zweiter Aufbewahrungsort.
**Gelöscht, ersatzlos:** `tools/chemenu/sections.py` komplett, `heading_re`, der
Alias-Mechanismus, `PRE_CONVENTIONS_NAMES`, `cite_block_heading`, `provenance.__getattr__`, die
`{section.*}`-Template-Variablen, `xref`s Abschnittssuche. Kein Überschriftentext liegt mehr in
Python - bis auf zwei kosmetische Fallbacks, und die sind harmlos geworden: der Marker trägt die
Identität, also rendert ein falscher Default falsche Wörter statt Struktur zu zerlegen, und der
nächste Write repariert es.
**Kanten sind direktional, und das war keine Geschmacksfrage.** Die per-Collection-Autorisierung
ist mit einer automatisch gespiegelten Gegenkante logisch unverträglich: die Spiegelhälfte
entsteht in einer Collection, deren Regeln der Autor nie gelesen hat. Entweder schriebe das
Werkzeug unautorisierte Kanten, oder die Regel "die Quellcollection entscheidet" löst sich auf.
Der Navigationseinwand wird dabei *besser* beantwortet als vorher: `wikitool links show --page`
berechnet die Eingangssicht über den Korpus, vollständig und ohne Pflege, und das gerenderte
Bullet ist ein gewöhnlicher `[[wikilink]]` - ein Backlink-Panel zeigt es ohnehin. Die erzwungene
Gegenkante garantierte nie Vollständigkeit, nur dass jemand daran gedacht hat.
**Der Orphan-Check meldet dadurch mehr,** und das ist die Prüfung bei der Arbeit: sie misst jetzt
Erreichbarkeit statt "ist `xref` gelaufen".
**`obligation:` trennt zwei Achsen, die vorher eine waren.** `migration_kind:` sagt *wie*
gearbeitet wird, neu `obligation: required|offered` *ob* überhaupt. Eine `offered`-Migration ist
ein Angebot für eine Datei, die der Instanz gehört - sie blockiert nie, steht nicht in der Kette,
und `migrate done` verbucht sie im Ledger, **ohne** `kb_version` zu bewegen. Genau daran hing ein
Entwurfsfehler, den erst der Test gezeigt hat: Offers gegen `kb_version` zu filtern hätte jede
Offer verschwinden lassen, sobald irgendein unbeteiligter Pflichtschritt lief. Dazu ist die
Erkennungshälfte aktiviert, die seit ihrer Einführung ungelesen dalag - die sha256 pro Datei in
`.wikitool-release.json` beantwortet jetzt "editiert oder nur empfangen", also ob eine Offer
kopiert werden darf oder von Hand abgeglichen werden muss.
**`types/` teilt sich entlang `root:`.** `root: kb` heißt Wissensseite heißt Instanz: die vier
Page-Type-Specs samt Schemas gehen als `.template`, `instruction`/`lint-report`/`type-spec`
verbatim. Damit ist die deutsche Prosa in jenen vier Dateien **korrekt statt Migrationsschuld** -
es war die richtige Sprache an einem Ort mit falsch deklariertem Eigentümer. Was der Stack von
der Type-Schicht noch verlangt, ist eine Zeile: ein Type-Spec `name: source`, dessen Schema
`raw_files` fordert. `STACK_REQUIRED_COLLECTIONS` entfällt als separate Liste - die pflichtige
Collection wird aus dem `base_dir` dieses Typs abgeleitet.
**Warum das MAJOR ist.** Vorwärts: `sections:` hat eine andere Form, `outbound:` fehlt, und die
in 3.0.0 übernommenen Page-Type-Templates enthalten `{section.*}`-Variablen, die es nicht mehr
gibt - `new` schriebe sie wörtlich in neue Seiten. Rückwärts: 4.0.0 schreibt gelabelte Kanten,
die 3.0.0s Schema als `type: string` ablehnt. Beide Hälften des Drop-in-Tests fallen.
**Der Korpus dieser Instanz ist noch nicht umgestellt.** Diese Version liefert die Maschinerie;
`lint` meldet die 480 noch ungelabelten Kanten als Findings, nicht als Fehler, weil das genau das
Fenster ist, für das `.wikitool-kb.json` existiert. `malformed_edges` und `unbalanced_markers`
sind dagegen sofort hart - keines beschreibt eine unkonvertierte Seite, nur eine kaputte. Die
Beförderung der beiden anderen kommt, wenn der Korpus sie bestehen kann.
---
## 3.0.0 - 2026-09-02 - Autorenkonventionen nach Eigentum geschnitten: kb/CONVENTIONS.md, deklarierte Collections
**Author:** Torben Nehmer
**Breaking Change:** kb/CONTRACT.md ist um alles gekuerzt, was eine Instanz selbst entscheidet; das steht jetzt in einer neuen, instanzeigenen kb/CONVENTIONS.md, aus der der Compiler die drei toolgefuehrten Abschnittsnamen liest. Eine bestehende Instanz muss diese Datei anlegen, auf jedem kb/*/COLLECTION.md profile: und required_by_stack: deklarieren und kb/CONTRACT.md aus dem Release nachziehen - sonst FAILt doctor und docs verify bricht. Ablauf: instructions/migrations/3.0.0-authoring-conventions.md
`kb/CONTRACT.md` war eine Datei mit zwei Autoritäten. Der eine Teil ist code-erzwungen und in
jeder Instanz gleich; der andere - **§ Language komplett**, das Beziehungslabel-Vokabular, die
Tonfall-Beispiele samt deutscher Buzzword-Liste, die Confidence-Rubrik, das ADR-Präfix - ist
Konvention, die jede Instanz für sich entscheidet, und wurde trotzdem als bindender Contract
verbatim ausgeliefert. Wer bei Schritt 5 von `setup-instance.md` "Englisch" antwortete, hatte
danach `kb/CONTRACT.md`, vier Type-Specs **und `tools/chemenu/sections.py`** lokal geändert -
und `private-instance.md`s Decision Point sagt für so einen Merge-Konflikt: Upstream-Seite
nehmen. Für diese Instanz hieß das: KB-Sprache zurück auf Deutsch.
**Der Schnitt läuft jetzt danach, wer den Satz ändern darf.** `kb/CONTRACT.md` behält, was
`wikitool` erzwingt; neu daneben liegt `kb/CONVENTIONS.md`, die **genauso bindet** und der
Instanz gehört. Unterschied ist Eigentum, nicht Autorität - deshalb liefert die Distribution nur
`kb/CONVENTIONS.md.template`, exakt der `USER.md`/`SOUL.md`-Split ein Verzeichnis tiefer. Dazu
`instructions/kb-profiles.md`: der Katalog erprobter Profile, ausdrücklich **Palette und kein
Enum**. Übernommen wird der *Text* in die Instanzdatei, nie ein Verweis auf den Katalog - ein
Verweis wäre wieder genau die Konstruktion, die dieser Release beendet.
**`sections.py` hält keine Überschrift mehr.** `RELATIONSHIPS = "Beziehungen"` war die Stelle,
an der die Konvention in Code übergelaufen war: solange sie dort stand, konnte kein Template die
Sprache umstellen. Neu ist `tools/chemenu/conventions.py`, das die drei Namen aus
`kb/CONVENTIONS.md` liest; `sections.py` löst sie per PEP 562 bei jedem Zugriff auf, wie
`config` seine Pfade - ein Modulkonstante hätte den Wert an den Baum gebunden, in dem der Prozess
gestartet ist. Aus demselben Grund ist `provenance.CITE_BLOCK_HEADING` ein `__getattr__` und
`render_cite_block(heading=None)` löst innerhalb des Aufrufs auf. Der Alias-Mechanismus, den das
Modul schon hatte, **ist** der Migrationspfad: erkannt wird die kanonische Form plus die
deklarierten `section_aliases:` plus das, was dieser Stack vor der Konventionsdatei geschrieben
hat. Ohne Datei antwortet dieser Fallback - richtig für jeden Korpus, der ihn erreichen kann,
denn der wurde unter genau diesen Namen geschrieben; `doctor` ist die laute Hälfte davon.
**Die vier Page-Type-Specs schreiben `## {section.relationships}`** statt einer Überschrift.
Neue Template-Variablen `{section.relationships}` / `{section.see_also}` / `{section.footnotes}`,
gefüllt aus der Instanzdeklaration. Damit ändert eine anderssprachige Instanz **keine Datei unter
`tools/` oder `types/`** mehr - was Schritt 5 von `setup-instance.md` von fünf Editierstellen
über drei Schichten auf eine Entscheidung reduziert.
**`COLLECTION.md` bekommt Frontmatter.** Bisher wurde eine Collection rein an der Dateipräsenz
erkannt; die Deklaration brauchte einen Träger, sonst wäre der Ortsschnitt nur durch einen
Prosaschnitt ersetzt worden. `profile:` nennt den übernommenen Katalogeintrag (Freitext - eine
selbst angelegte Collection hat dort keinen), `required_by_stack:` sagt, ob `wikitool` die
Collection *namentlich* auflöst. Das zweite ist **nicht** die Wahl der Instanz: `docs verify`
prüft es beidseitig gegen `kb_collections.STACK_REQUIRED_COLLECTIONS`. Heute steht dort genau
`sources` - `sources coverage`, die `[^cite-id]`-Auflösung und `kb/provenance.md` hängen an dem
Namen, `entities` an keinem.
**Das zweite Leck der Merge-Prozedur ist zu.** `git checkout HEAD -- kb raw` holte *alles* unter
beiden Stages auf den Vor-Merge-Stand - auch `kb/CONTRACT.md` und `raw/CONTRACT.md`. Änderte der
Upstream einen davon, warf die Prozedur das Update still weg, und die Kontrollzeile meldete dabei
*leer*, bestätigte den Fehler also, statt ihn zu fangen. `private-instance.md` nimmt die
Upstream-Seite jetzt für die drei Maschinerie-Pfade unter den Content-Stages zurück
(`kb/CONTRACT.md`, `kb/CONVENTIONS.md.template`, `raw/CONTRACT.md`) und schließt sie aus der
Kontrollzeile aus. Dieselbe Altlast in der Tarball-Richtung: `INSTALL.md` Schritt 3 fasste `kb/`
gar nicht an und zog `kb/CONTRACT.md` damit nie nach - jetzt ausdrücklich benannt.
**Verworfen, gemessen: `sources/` aus `kb/` herausziehen.** Der Graph ist einwurzelig
(`kb_scan.iter_kb_pages` macht ein `rglob` über `kb/`, darauf sitzen Link-Graph, Orphan-Check,
`index rebuild` und `search`), und Source-Seiten sind darin der dichteste Knotentyp. Ein Hoist
machte jede Graph-Operation dauerhaft zweiwurzelig, um ein Verzeichnis umzubenennen. Vor allem
aber kann der *Ort* Eigentum ohnehin nicht kodieren, sobald Collections offen sind: eine selbst
angelegte liegt im selben `kb/` wie die Defaults. Eigentum ist eine deklarierte Eigenschaft -
daher das Frontmatter oben. Gitea #39 trägt die Ablehnung im Volltext.
**Warum das MAJOR ist.** Die Rückwärtshälfte des Drop-in-Tests hält - 2.5.0 ignoriert beide neuen
Deklarationen folgenlos. Die Vorwärtshälfte nicht: nach dem Kopieren der Maschinerie FAILt
`doctor` auf der fehlenden `kb/CONVENTIONS.md`, `docs verify` bricht auf den undeklarierten
Collections, und `kb/CONTRACT.md` muss aus dem Release nachgezogen werden. Ein Shim war die
Alternative (`doctor` nur WARN, Pflichtfelder tolerant) und wurde verworfen: er hätte genau den
Zustand normalisiert, in dem eine Instanz glaubt, sie habe entschieden, während in Wahrheit der
Fallback antwortet - für eine englische Instanz hieße das `## Beziehungen` in englischen Seiten.
Die Handarbeit ist eine Datei und zwei Frontmatter-Zeilen je Collection; keine einzige `kb/`-Seite
ändert sich, weshalb `migrate done 3.0.0 --pages 0` ehrlich und kein Platzhalter ist.
---
## 2.5.0 - 2026-09-02 - Versionsstelle: Kompatibilitaet statt Inhaltsmigration, Breaking-Change-Vermerk erzwungen
**Author:** Torben Nehmer
Die Doku des Stacks führte für die Wahl der Versionsstelle zwei Fragen zusammen, die nicht
dieselbe sind: *muss der Korpus migriert werden?* und *ist die neue Version ein
Drop-in-Ersatz?* An allen drei Stellen, die einer Session vor dem Bump begegnen - die Tabelle
in `stack-dev`, der Modul-Docstring in `version.py`, `INSTALL.md` § Version und Updates -
stand nur die erste. Nur `version bump --help` formulierte es richtig, also ausgerechnet die
Stelle, die man erst liest, wenn die Entscheidung gefallen ist. Bei `2.0.0` hat genau diese
Lücke zu `1.9.0` geführt (Gitea #26; der Eintrag zu `2.0.0` trägt den Fall im Volltext).
**Die Regel, jetzt an einer Stelle.** Neu ist
`instructions/dev/version-parts.md`: der Drop-in-Test in zwei Hälften - vorwärts (kopieren,
fertig, keine Handarbeit durch Nutzer oder Migrationsskript) und rückwärts (die alte Version
lässt sich noch zurückinstallieren) -, der Katalog der Brüche, die die Grenze bei völlig
unangetastetem `kb/` überschreiten (Update-Pfad, Artefaktname, Import-Name, Flags, Envvars,
Shape maschinengelesener Dateien), und der Fall `2.0.0` als Fallbeispiel. `instructions/dev/`,
also nicht in einer ausgelieferten Instanz - die Datei entscheidet, wer den Stack entwickelt.
Die ausgelieferten Artefakte tragen deshalb die Kurzform der Regel, aber keinen Verweis auf
sie: `instructions verify` meldet einen solchen Verweis, weil er nach `dist export` ins Leere
zeigen würde.
**Breaking Changes sind teuer, also genehmigungspflichtig.** `stack-dev` hat einen
Entscheidungspunkt bekommen: stellt sich eine Änderung als nicht drop-in heraus, wird nicht
aus eigener Initiative über die Grenze gebumpt. Vorgelegt werden der konkrete Bruch, was jede
bestehende Instanz von Hand tun muss, und die Alternativen - Bruch per Shim vermeiden,
aufschieben und mit dem nächsten bündeln, oder in einen kompatiblen und einen brechenden Teil
mit Deprecation-Fenster spalten -, dazu eine Empfehlung. Dann Freigabe abwarten.
**`--breaking` ist neu und bei jedem Grenzübertritt Pflicht.** Es schreibt eine
`**Breaking Change:**`-Zeile in den Eintrag, vor die Migrationszeile: das ist, was der Operator
einer bestehenden Instanz beim Lesen der Release-Notes zuerst braucht. `docs verify` prüft den
neuesten grenzüberschreitenden Eintrag darauf, analog zu `**Migration:** none required` und aus
demselben Grund - eine Prosa-Regel driftet, ein Marker nicht. Auf einem kompatiblen Bump wird
`--breaking` verweigert, damit es keine Gewohnheit wird. Beide Prüfungen bleiben getrennt:
`check_migration_for_boundary` fragt, ob der Inhalt sich bewegt, `check_breaking_change_for_boundary`,
ob dem Operator gesagt wurde, dass der Tausch keiner zum Drüberkopieren ist. Ein Rebranding
besteht die erste und fällt durch die zweite.
**Was ausdrücklich nicht passiert:** `docs verify` prüft weiterhin nicht, *ob die Stelle
richtig gewählt* wurde. Kein Validator unterscheidet ein umbenanntes Flag von einem neuen; das
bleibt Urteil, und dieser Eintrag existiert, weil das Urteil eine Anleitung braucht, keinen
Automaten.
Berührt: `instructions/dev/version-parts.md` (neu), `instructions/dev/stack-dev/SKILL.md`,
`tools/chemenu/version.py`, `tools/chemenu/commands/version_cmd.py`,
`tools/chemenu/commands/docs_verify.py`, `tools/CONTRACT.md`, `INSTALL.md`. Sechs neue Tests.
---
## 2.4.1 - 2026-09-02 - INSTALL-MCP.md: Menschendoku fuer den Leseserver, und sie wird mit ausgeliefert
**Author:** Torben Nehmer
Nachtrag zu 2.4.0. Der Server war ausgeliefert und für Menschen nur über `tools/CONTRACT.md` und
eine Instruktion beschrieben — beide für Agenten geschrieben. `INSTALL-MCP.md` schließt das,
neben `INSTALL.md` und in derselben Form: Voraussetzungen, sechs Schritte, Verifikation,
Troubleshooting.
**Was drinsteht, das sonst nirgends zusammen stand:** die Client-Konfiguration als JSON-Block;
dass `WIKI_TRACE=0` nicht optional ist und wie die Startverweigerung aussieht; dass
`--host 0.0.0.0` bewusst nicht der Default ist; und der Verweis auf die
Traefik-ForwardAuth-Middleware unter <https://gitea.nehmer.net/torben/gitea-mcp-forward-auth>
samt dem, was sie erwartet (Bearer-Token gegen SHA-256-Hashes, `AUTH_PROXY_TOKEN_HASHES_DIR`).
Der Verifikationsabschnitt ist ein lauffähiges Skript — wörtlich so ausgeführt, bevor es hier
landete.
**`dist export` liefert die Datei mit.** `ROOT_FILES` ist eine Allowlist, also hätte sie sonst
keine Instanz erreicht: ein ausgelieferter Server, dessen Bedienungsanleitung im Ursprungs-Repo
zurückbleibt, ist einer, den der Empfänger durch Quelltextlesen findet. `INSTALL.md` bekommt
zusätzlich `CHEMENU_ROOT` und `WIKI_TRACE`/`WIKI_TRACE_DIR` in seine Konfigurationstabelle —
beide waren seit 2.3.0 bzw. seit jeher wirksam und dort nicht aufgeführt.
**Dateien:** `INSTALL-MCP.md` (neu), `INSTALL.md`, `README.md`,
`tools/chemenu/commands/dist_cmd.py`.
---
## 2.4.0 - 2026-09-02 - MCP-Leseserver: zweiter Konsument auf demselben Kern
**Author:** Torben Nehmer
Letzter Schritt der Sequenz aus #36, inhaltlich Issue #19. `chemenu` bekommt einen zweiten
Konsumenten: `search`, `types`, `describe_type`, `lint` und `status` über MCP. Kein CLI mit
angeschraubter Netzwerkschnittstelle — CLI und Server sind zwei Adapter auf dem Kern, den 2.3.0
freigelegt hat.
**`tools/chemenu/mcp/`**, im Repo statt als eigenes Artefakt. Der Golden-Test, der die
Serverantworten gegen die `--json`-Formen der CLI hält, läuft nur mit beiden Seiten in einer
Testsuite; getrennt würde aus einem Contract eine Versionsabsprache. Der Test ruft `wikitool` als
Subprozess gegen denselben Baum auf, über `CHEMENU_ROOT` — womit er nebenbei die Root-Auflösung
von außen mitprüft.
**Zwei Transports.** `stdio` zum Entwickeln und Testen ohne Netz, `streamable-http` für die
Auslieferung — der einzige, vor den sich die Authentifizierungs-Middleware überhaupt setzen kann,
weil sie ein HTTP-Reverse-Proxy ist. `sse` ist über das SDK erreichbar und wird bewusst nicht
angeboten: der abgelöste Remote-Transport, jetzt darauf zu bauen verschiebt den Wechsel nur.
`--host`/`--port` gibt es, weil der Default auf Loopback bindet und ein Container hinter einem
Proxy eine Adresse braucht, die der Proxy erreicht — eine Eigenschaft der Software, nicht einer
Installation. Beide Transports sind gegen den echten Korpus gegengeprüft.
**Kein Schreibpfad, strukturell.** Weder der Server noch `chemenu.api` importiert irgendetwas
unter `chemenu.commands`, also existieren `new`, `touch`, `xref`, `cite`, `publish`, `migrate`
und `version bump` in dieser Reichweite gar nicht, statt aus einer Liste gefiltert zu werden. Ein
Test importiert das Servermodul in einem frischen Interpreter und sieht in `sys.modules` nach;
ein zweiter ruft alle fünf Tools auf und vergleicht den Dateibaum, `HEAD` und
`git status --porcelain` vorher/nachher.
**Jede Antwort trägt ihren Commit.** `commit` und `as_of` in jedem Payload; `null` heißt, der
bediente Baum hat uncommittete Änderungen und die Antwort entspricht keiner Revision. Der Stempel
ist die Revision, aus der die Seiten *tatsächlich* gelesen wurden — zwischen Laden und Stempeln
kann der Baum sich bewegen, deshalb reicht der Ladepfad seine Revision durch, statt noch einmal
zu fragen. Das war beim ersten Durchlauf falsch: `types`/`lint`/`status` lasen die zuletzt
*gecachte* Revision und stempelten `null`, obwohl der Baum sauber war.
**Telemetrie in den bedienten Baum wird beim Start verweigert**, nicht stillschweigend
umgeleitet. Tracing ist per Default an und schreibt nach `reports/telemetry/` im Repo — genau das
Verzeichnis, das der Sync per `git reset --hard` wegräumen darf. `WIKI_TRACE=0` oder
`WIKI_TRACE_DIR` außerhalb des Korpus. Heute schreibt auf diesem Pfad nichts (der Emitter hängt an
`cli.main()` und den Gates), die Sperre ist gegen später.
**Fehler an der Protokollgrenze.** Ein `ChemenuError` wird zum `ToolError` des SDK — eine
absichtliche Ablehnung, deren Text den Aufrufer erreicht. Alles andere bleibt ein Absturz, dessen
Text auf dem Server bleibt. Ein kaputtes Prädikat ist das Argument des Aufrufers, also muss die
Zeile mitreisen, die sagt, was stattdessen zu schreiben ist.
**Bewusst nicht enthalten:** Authentifizierung und Rate Limiting (Middleware vor dem Prozess),
Deployment (private Infrastruktur), der Iteration Budget Gate — er begrenzt eine Agenten-Session
und nicht einen Nutzer, weshalb Retrieval von ihm befreit ist; ihn hier als Rate Limiter zu
benutzen würde ihn dazu verwässern.
**Die Abhängigkeit ist optional** (`tools/requirements-mcp.txt`): eine Instanz, die nur die CLI
benutzt, soll dafür nicht pydantic, starlette, uvicorn und cryptography installieren müssen. CI
installiert sie, denn ein übersprungener Golden-Test ist genau der Weg, auf dem Server und CLI
unbemerkt auseinanderlaufen.
Betrieb und Sync-Mechanismus: [instructions/mcp-read-server.md](instructions/mcp-read-server.md).
Polling (`git fetch && git reset --hard`) statt Webhook — kein eingehender Endpunkt, keine
Signaturprüfung. `reset --hard` ist dort tragend und keine Bequemlichkeit: ein abgedrifteter Baum
antwortet zwar richtig, parst aber bei jeder Anfrage neu und stempelt jede Antwort mit `null`.
**Dateien:** `chemenu/mcp/` (neu: `server.py`, `__main__.py`), `chemenu/api.py`,
`tools/requirements-mcp.txt` (neu), `instructions/mcp-read-server.md` (neu), `tools/CONTRACT.md`,
`tools/README.md`, `.gitea/workflows/ci.yml`, `tests/test_mcp_server.py` (neu).
---
## 2.3.0 - 2026-09-02 - Bibliotheksgrenze: chemenu laesst sich auf einen Korpus zeigen
**Author:** Torben Nehmer
Dritter Schritt der Sequenz aus #36, inhaltlich Issue #31. Der Schritt, der `chemenu` von einem
Skript, das in seinem eigenen Verzeichnis lebt, zu einer Bibliothek macht, auf die man einen
Korpus *zeigen* kann. Rückwärtskompatibel: ohne gesetzte Variable verhält sich `tools/wikitool`
unverändert.
**Root-Auflösung nach Präzedenz statt nach Dateilage.** `config.resolve_root()`: expliziter
Parameter → `$CHEMENU_ROOT` → Walk-up wie bisher. Der Walk-up bleibt Default. Wichtiger als die
neue Fähigkeit ist die beseitigte Fehlerklasse: `ROOT` und alles darunter waren
Modulkonstanten, also **zur Importzeit gebunden**, und
`monkeypatch.setattr(config, "ROOT", ziel)` zeigte `ROOT` um, aber nicht `KB_DIR`/`RAW_DIR`. Wer
sich darauf verließ, baute etwas, das scheinbar auf einem Zielbaum arbeitete und in Wahrheit aus
dem Entwickler-Checkout antwortete. Die abgeleiteten Pfade werden jetzt bei jedem Zugriff
aufgelöst (PEP 562 `__getattr__`) und folgen `ROOT` — der halb-umgezeigte Zustand ist nicht mehr
konstruierbar. `CHEMENU_ROOT` ist in `_WIKITOOL_ENV` registriert, #23-konform.
**`config.reset()` und `config.rooted()`.** `reset()` nimmt Zuweisungen auf die verwalteten
Pfadnamen zurück; die Testsuite ruft es zwischen Tests, und das ist dort nicht optional:
`monkeypatch` merkt sich den alten Wert, indem es ihn *liest* — also auflöst — und schreibt ihn
beim Aufräumen als echtes Attribut zurück. Genau die stale Bindung, die der Umbau unmöglich
machen sollte, vom Cleanup wieder aufgebaut. `rooted(root)` setzt den Root für die Dauer eines
Blocks, prozessweit und damit nicht thread-sicher — der Aufrufer hält das Lock, dieselbe
Disziplin wie beim Korpus-Cache. Nötig, weil nicht alles einen Root als Argument nimmt: der
modulglobale `TypeResolver` muss `types/` finden, und ohne ihn läse ein fremder Korpus mit den
Type-Specs *dieses* Checkouts.
**Die Naht ist gezogen.** `run_search`/`run_lint`/`types` lagen in Modulen, die `typer` auf
Modulebene importieren und über `_util` auch `rich` — wer sie in-process aufrief, zog den
kompletten CLI-Kopf mit. Der reine Kern liegt jetzt in `search/service.py`, `lint_core.py` und
`types_core.py`; `commands/` sind die Terminal-Adapter darüber und re-exportieren die Namen, damit
kein bestehender Import bricht. Ein Test importiert `chemenu.api` in einem frischen Interpreter
und prüft, dass weder `chemenu.commands.*` noch `typer`/`rich`/`click` geladen werden.
**`chemenu.api.Corpus` als In-Process-Einstieg.** Nimmt einen Root, liefert exakt die
`--json`-Formen der CLI zurück — ein Wire-Contract statt zwei — und stempelt jede Antwort mit dem
Commit-SHA und einem Zeitstempel (`commit`, `as_of`), so dass aus einer stillen veralteten Antwort
eine sichtbare wird. `search`/`lint`/`types`/`describe_type`/`status`; `status` ist bewusst
serverseitig **komponiert** und kein Wrapper, weil es kein `wikitool status` gibt. Lesend
strukturell: nichts unter `chemenu.commands` wird importiert, die Schreibfunktionen existieren in
dieser Oberfläche also gar nicht, statt gefiltert zu werden. Das ist die Grenze, auf der #19
aufsetzt.
**Exceptions statt Exit-Codes an der Grenze.** `chemenu/errors.py`: `ChemenuError` mit
`ValidationError` (Eingabe abgelehnt) und `BackendError` (Abhängigkeit fehlt oder scheitert).
`PredicateError`, `FrontmatterError`, `UnknownBackend` und die beiden `Ripgrep*` hängen jetzt
darunter; `ValidationError` erbt zusätzlich von `ValueError`, weil `PredicateError` vorher eines
war und Aufrufer es so fangen. Das CLI-Verhalten ist unverändert: `fail()``ERROR`-Zeile,
Exit 1, Budget-Refund.
**`resolve()` reicht den Root an das Backend durch.** Vorher konnte ein Aufrufer `run_search` einen
Korpus übergeben, während `RipgrepBackend` weiter `config.KB_DIR` durchlief — die Anfrage aus dem
einen Baum beantwortet, die Seiten aus dem anderen gelesen, ohne dass irgendetwas das gesagt
hätte.
**Zwei Abhängigkeiten, die durch Zufall hielten, stehen jetzt da.** `TypeResolver.repo_root`
folgt `ROOT`, statt beim Import zu binden — womit Fixtures, die `ROOT` auf einen tmp-Baum zeigen,
die mitgelieferten Type-Specs ausdrücklich benennen müssen (`use_shipped_type_specs`). Dieselbe
Form wie das Loch, für das `raw_dir` geschrieben wurde, eine Ebene tiefer. Und
`types describe --json` trug `root:` nicht im Payload, obwohl `types list --json` es tut:
`types/instruction.md` deklariert `root: repo`, die Renderer-Zeile las es direkt aus dem
Frontmatter. Jetzt im Payload, in derselben Form wie bei `list`.
**Dateien:** `config.py`, `errors.py` (neu), `api.py` (neu), `search/service.py` (neu),
`lint_core.py` (neu), `types_core.py` (neu), `search/registry.py`, `type_resolver.py`,
`commands/search.py`, `commands/lint.py`, `commands/types_cmd.py`, `tools/CONTRACT.md`, dazu
`tests/conftest.py`, `tests/test_api.py` (neu), `tests/test_new_page.py`,
`tests/test_instructions_cmd.py`.
---
## 2.2.4 - 2026-09-02 - Haertung des Lesepfads: ReDoS, Subprozess-Timeout, YAML-Alias-Budget, Korpus-Cache
**Author:** Torben Nehmer
Zweiter Schritt der Sequenz aus #36, inhaltlich Issue #33. Limits vor der Exposition: der
Lesepfad bekommt mit dem MCP-Server (#19) einen Konsumenten, der nicht der Operator ist. Alle
sechs Befunde waren reproduziert, nicht vermutet; jeder hat jetzt eine Regression.
**ReDoS über `--regex` beseitigt.** `search/ripgrep.py:_contains` gab nutzergesteuerten Regex an
Pythons Backtracking-Engine: `(\w+\s?)+$` gegen 114 Zeichen gewöhnlichen Seiteninhalts terminiert
nicht in acht Sekunden, ein deterministisch scheiterndes Muster braucht 0,2 ms — der Unterschied
ist das Muster, nicht der Heuhaufen. `build_hit` ruft die Funktion je Treffer zweimal auf, und
`\w` matcht jede Seite: eine Anfrage kaufte zwei unbegrenzte Suchen pro Korpusseite. Der Zweig ist
**gelöscht**, nicht begrenzt. `rg` hat das Muster mit einer linearen Engine längst angewendet, wenn
die Funktion überhaupt läuft; verloren geht nur der zusätzliche Titel-/Summary-Bonus für
nicht-literale Muster, und Summary wie H1 sind selbst Zeilen in der Datei, die `rg` zählt.
**Subprozess-Timeout.** `rg` wird nach 30 s abgeräumt und über den vorhandenen
`RipgrepFailed`-Pfad gemeldet. Kein Performance-Budget — eine Fixed-String-Suche kostet hier 6 ms —
sondern ein Hänger-Abbruch, damit ein Aufruf als Fehler endet statt den Aufrufer offenzuhalten,
während seine Ausgabe in den Heap puffert.
**YAML-Anchors und -Aliases im Frontmatter werden verweigert, nicht budgetiert.** Gemessen:
267 Byte werden in 0,2 ms zu 672.603 Knoten beim Traversal, Wachstum 9ⁿ bei konstanter Parse-Zeit
— ein Größenlimit fasst das nicht an, weil die Eingabe klein bleibt. Die Prüfung läuft auf dem
*Event*-Strom (`yaml.parse`), der nichts auflöst, kostet also O(Text) und löst nie aus, wonach sie
fragt; `*` ist in jedem Alias-Knoten notwendig, seine Abwesenheit beweist Abwesenheit ohne jeden
Parse — der Weg, den jede echte Seite nimmt. Dazu ein Größenlimit von 64 KiB und ein Abfangen von
`RecursionError` (PyYAML komponiert rekursiv, tiefe Verschachtelung ist kein `YAMLError`). Heute
nicht erreichbar, weil `kb/` der Operator committet; erreichbar mit der Ingest-Queue (#32).
**`CSafeLoader` statt `SafeLoader`, mit Fallback.** Gemessen über diesen Korpus (176 Seiten,
best of 5): **265 ms → 54 ms**. Kein Mikro-Tuning — der Korpus-Parse war der größte Einzelposten
eines `search`-Aufrufs und skaliert linear mit der Korpusgröße. End-to-end fällt ein
`wikitool search` damit von 593 ms auf **347 ms**; die verbleibenden 262 ms sind Modulimport und
entfallen erst im residenten Prozess (#19).
**Stiller Frontmatter-Verlust wird gemeldet.** Entschieden: der Lesepfad *nennt* die Seite, statt
sie zu schlucken. Kaputtes YAML wird weiterhin zu `{}` — Massenoperationen dürfen an einer Seite
nicht scheitern —, aber der Grund wird mitgeführt (`Page.frontmatter_error`) und ausgegeben:
`search --json` trägt immer eine `unreadable`-Liste aus `{path, reason}`, die Tabellenform
schreibt dieselben Zeilen nach stderr. Das war nötig, weil so eine Seite weder `confidence` noch
`kind` hat und damit aus jedem positiven `--field`-Prädikat fällt — ausgerechnet aus der
Low-Confidence-Suche, die Seiten in genau diesem Zustand finden soll — und dabei aussieht wie eine
Seite, die nicht gematcht hat. Für Frontmatter, das diese Instanz nicht selbst geschrieben hat,
steht `read_page_strict()` bereit: die Quarantäne aus #32 muss strikt lesen, wo ein
unlesbares Dokument das Dokument stoppen und nicht leeren soll.
**Ein Parser statt zwei.** `read_page()` und `frontmatter_error()` liefen bisher getrennt durch
`safe_load` — so konnte der permissive Weg zu `{}` degradieren aus einem Grund, den der strikte
Weg anders beschrieb, und jeder Aufrufer, der beide Antworten wollte, las die Datei zweimal.
Beide gehen jetzt durch `_load_frontmatter()`; ein Test hält sie gegeneinander.
**Korpus-Cache am Commit-SHA** (`chemenu/corpus_cache.py`), als Objekt, das ein Aufrufer *hält*
kein Modul-Dict, das sich hinter allen einschaltet. Die CLI hält keins und verhält sich unverändert
(ein Aufruf pro Prozess, nichts wiederzuverwenden); der residente Prozess aus #19 hält eins.
Entscheidend ist nicht die Geschwindigkeit, sondern dass nichts veraltet: **ein schmutziger
Arbeitsbaum wird nie gecacht**, sonst bekäme eine Sitzung, die eine Seite schreibt und danach
sucht, die Analyse von vor dem Schreiben — bei unverändertem SHA. Kann git nicht antworten, gilt
der Baum als schmutzig. Derselbe SHA ist der Antwort-Stempel aus #19, per Konstruktion also die
Revision, aus der die Antwort auch wirklich berechnet wurde.
**Erhalten geblieben** ist die Eigenschaft, die kein Befund war: kein `shell=True`,
`--fixed-strings` als Default, `--`-Terminator. Der Modul-Docstring führt sie jetzt als drei
tragende Zusagen statt zwei — die dritte ist, dass nutzergesteuerte Muster ausschließlich `rg`
sieht.
**Dateien:** `frontmatter_io.py`, `search/ripgrep.py`, `commands/search.py`, `page.py`,
`corpus_cache.py` (neu), `tools/CONTRACT.md`, dazu `tests/test_frontmatter_io.py`,
`tests/test_search.py`, `tests/test_corpus_cache.py` (neu).
---
## 2.2.3 - 2026-09-02 - Publish-Remote-Gate in diesem Checkout scharf, doctor benennt den Zustand
**Author:** Torben Nehmer
Erster Schritt der Sequenz aus Issue #36 (Weg zum MCP-Leseserver), inhaltlich Issue #34. Das
Gate war seit 2.2.0 implementiert und in diesem Checkout **inert**: `.wikitool-remotes.json`
fehlte, und eine fehlende Datei heißt unbeschränkt. `ENVIRONMENT.md` beschrieb `origin`
gleichzeitig als einziges Publish-Ziel, `AGENTS.md` führt das Gate unter den drei in Code
erzwungenen Grenzen. Eine dokumentierte Sicherung, die nicht greift, ist schlechter als eine
fehlende — sie erzeugt genau das Vertrauen, das sie nicht verdient.
**Die Datei ist angelegt** (gitignored, per Checkout, reist nicht mit) und gegengeprüft: ein
`publish --remote` auf ein nicht gelistetes Ziel verweigert mit Exit 42, bevor irgendetwas
gestaged wird, und der Arbeitsbaum bleibt unberührt. Damit steht die Sicherung **vor** dem Klonen
der privaten Instanz (#30) — nachträglich gesetzt ließe sie genau das Fenster offen, das sie
schließt.
**`doctor` sagt jetzt, ob das Gate scharf ist, nicht nur ob die Datei da ist.** Alle drei
Zustände beginnen mit `Gate armed:` bzw. `Gate not armed:`; der einzelne Remote ohne Allowlist
bleibt `OK` (er hat nichts zu schützen, und ein FAIL machte die Datei durch die Hintertür
verpflichtend), sagt aber ausdrücklich, dass jedes Push-Ziel durchkommt. Der Fall, der wirklich
beißt — mehrere Remotes ohne Allowlist — bleibt `WARN`. Der Check hatte bislang **keine Tests**;
drei sind dazugekommen, einer je Zustand.
**Dateien:** `.wikitool-remotes.json` (neu, nicht committet), `doctor.check_publish_remotes()`,
`tools/chemenu/tests/test_doctor.py`.
---
## 2.2.2 - 2026-09-01 - Doku-Verdrahtung: Publish-Remote Gate im Werkzeugvertrag, Projektseite auf oeffentlich
**Author:** Torben Nehmer
Nachziehen dessen, was 2.2.0 und die Veröffentlichung offen gelassen haben. Gefunden durch eine
Durchsicht auf lose Enden, nicht durch einen Fehlschlag — `docs verify` deckt den Fall nicht ab,
weil es Kommando-*Namen* gegeneinander prüft, nicht ob ein neuer Fehlerfall beschrieben ist.
**`tools/CONTRACT.md` kannte das Publish-Remote-Gate nicht.** Der Werkzeugvertrag ist die Stelle,
an der pro Kommando steht, was ein Fehlschlag bedeutet und ob ein Retry sicher ist — und
`publish` hatte seit 2.2.0 einen dritten Weg zu Exit 42, der dort nirgends stand. Ergänzt in
beiden Richtungen: in der Kommandozeile (URL statt Remote-Name, `pushurl` wird gelesen, fehlende
Datei heißt unbeschränkt, kaputte Datei ist ein Fehler) und im Fehlerkontrakt, wo der
entscheidende Unterschied zu den anderen beiden Gates steht — es gibt **keine** `--confirm`-Zeile,
die der Agent nachreichen könnte.
**`instructions/gates.md` verwies nicht auf die Prozedur, für die das Gate gebaut wurde.**
Jetzt verlinkt, mit dem Hinweis auf Schritt 4: Das Gate gehört vor den ersten `publish`, später
hinzugefügt schützt es das Fenster nicht, das es schließen soll.
**Die Projektseite beschrieb sich selbst falsch.** `kb/entities/projects/Chemenu.md` nannte
Chemenu ein „persönliches IT-Wissens-Wiki" mit dem Zweck „Persönliche IT-Wissensbasis" — seit
der Veröffentlichung schlicht unzutreffend, und es ist die Seite, die ein Fremder über das
Projekt liest. Neu gefasst: deterministischer Wissenskompiler, diese Instanz seit 2026-09-01
öffentlich als Testbett und Demo, Lizenz benannt.
Die historische Aussage über die monolithische `AGENTS.md` (~30 KB) **bleibt** — sie war zu ihrer
Zeit korrekt und ist belegt. Sie ist jetzt als Ausgangspunkt datiert statt als Gegenwart
formuliert, dieselbe Unterscheidung, die #29 für die Issue-Texte trifft: ein Pfad als Wegweiser
wird nachgezogen, ein Pfad als Beleg bleibt stehen und bekommt ein Datum.
**Dateien:** `tools/CONTRACT.md`, `instructions/gates.md`,
`kb/entities/projects/Chemenu.md`, `kb/entities/INDEX.md` (generiert).
---
## 2.2.1 - 2026-09-01 - private-instance: der Demo-Korpus wandert beim Merge doch mit - Prozedur korrigiert
**Author:** Torben Nehmer
`instructions/private-instance.md` behauptete in 2.2.0, ein `git merge upstream/main` löse
Änderungen am Demo-Korpus stillschweigend auf, weil die private Instanz ihn einmal gelöscht hat:
*deleted-in-ours, unmodified-in-theirs*. Das war **nicht gemessen, sondern angenommen** — und es
ist falsch. Ein Nachbau mit einem Upstream, der seinen Korpus bewegt, zeigt drei verschiedene
Verhalten:
| Upstream tut | `git merge upstream/main` tut |
|---|---|
| ändert eine Seite, die du gelöscht hast | `CONFLICT (modify/delete)` — und **lässt die Upstream-Fassung im Arbeitsbaum liegen**. Ein `git add -A` beim Auflösen holt die Demo-Seite zurück. |
| legt eine neue Seite an | staged sie **stillschweigend**. Kein Konflikt, keine Meldung. |
| löscht eine Seite, die du auch gelöscht hast | nichts. Der einzige harmlose Fall. |
Die mittlere Zeile ist die gefährliche, weil nichts sie ankündigt. Ein Upstream, der einen
Demo-Korpus ausliefert **und** ihn als Testbett benutzt, legt Seiten an — und jede einzelne
landet in der privaten Instanz und taucht dort in `lint`, `index`, `search` und
`confidence decay` auf. Genau diese Doppelnutzung beschreibt Issue #28.
**Korrigiert:** Die Update-Prozedur hält den Merge mit `--no-commit` offen, zwingt `kb/` und
`raw/` danach auf den eigenen Stand zurück (`git rm --cached`, `rm -rf`, `git checkout HEAD --`)
und schließt ihn erst dann. Solange der Merge offen ist, zeigt `HEAD` noch auf den Commit davor,
und genau das macht den Schritt sauber. Anschließend eine Kontrolle, die man nicht überlesen
kann:
```bash
git diff --name-only $BEFORE HEAD -- kb raw # muss leer sein
```
Das Rezept ist wörtlich so nachgespielt worden, wie es jetzt in der Datei steht — mit einem
Upstream, der gleichzeitig eine Seite ändert, eine anlegt, eine löscht und dasselbe unter
`raw/` tut. Ergebnis: Stack aktualisiert, nur eigener Inhalt übrig, Kontrolle leer,
Arbeitsbaum sauber.
**Auch die Decision Points korrigiert.** „Konflikt in `kb/` per Hand als *keep deleted*
auflösen" stand vorher da und ist der Rat, der in die Falle führt: `git add -A` committet die
Fassung, die git im Arbeitsbaum liegen gelassen hat.
**Dateien:** `instructions/private-instance.md`.
---
## 2.2.0 - 2026-09-01 - Publish-Remote Gate: publish schreibt nur an erklaerte Ziele
**Author:** Torben Nehmer
Der Stack bekommt sein drittes Gate. Die beiden bestehenden fragen, ob eine Änderung zu groß
ist und ob ein Rebase gefährlich ist. Dieses fragt, was darunter liegt: **ob das überhaupt das
richtige Repository ist.**
**Das Problem entsteht erst durch die private Instanz.** Ein Checkout mit eigenem Inhalt hat
typischerweise zwei Remotes — sein eigenes und das öffentliche Upstream, von dem er
Stack-Updates zieht. Git unterscheidet die beim Push nicht, also legt ein falsches `--remote`
einen privaten Korpus auf ein öffentliches Repository. Das ist nicht billig rückholbar, und
zwar nachweislich: Beim Veröffentlichen dieses Repos blieb die gesamte alte History nach dem
Force-Push per SHA abrufbar, bis auf dem Server die Reflogs verfielen und `git gc --prune=now`
lief. Ein Force-Push bewegt den Branch, nicht die Objekte.
**`.wikitool-remotes.json` nennt die erlaubten Push-URLs.** Nicht die Remote-*Namen*: Eine
Namensliste ließe ein `publish` durch, dessen `origin` umgebogen wurde, und genau das ist der
Fall, den das Gate fangen soll. Gelesen wird die `pushurl`, wenn der Remote eine setzt, denn
dorthin schreibt `git push` tatsächlich.
**Pro Checkout und gitignored**, aus demselben Grund wie `ENVIRONMENT.md`: Zwei Klone pushen an
zwei verschiedene Orte, eine committete Kopie würde einem privaten Klon also mitteilen, das
öffentliche Upstream sei ein legitimes Ziel für seinen eigenen Inhalt. **Fehlt die Datei, gilt
keine Beschränkung** — ein Checkout mit einem Remote und ohne Privates hat nichts zu schützen,
und eine Pflichtdatei würde aus einer Sicherung Papierkram machen. Eine *kaputte* Datei ist
dagegen ein Fehler und kein „keine Beschränkung": Eine beschädigte Sicherung darf sich nicht
wie eine abgeschaltete verhalten.
**Kein Token, keine Flagge.** Die anderen beiden Gates lösen sich mit einem `--confirm <token>`,
weil ihre Frage („ist diese Änderung richtig?") für genau ein Changeset beantwortbar ist. Dieses
fragt „gehört dieser Inhalt in jenes Repository?", und das ist eine stehende Eigenschaft des
Checkouts, kein Einzelfallurteil. Der Weg daran vorbei ist ein bewusster Edit des Nutzers.
Ein Agent, der die Datei anfasst, um an einer Verweigerung vorbeizukommen, öffnet ein Gate aus
eigenem Antrieb — Invariante 6.
**`doctor` meldet den Zustand** statt ihn zu erzwingen: OK mit Anzahl der Ziele, OK bei
Abwesenheit mit einem Remote, und WARN bei mehr als einem Remote ohne Allowlist — also genau in
der Form, die eine private Instanz annimmt, sobald sie das Upstream hinzufügt.
**Und die Prozedur, für die das Gate gebaut wurde.** `instructions/private-instance.md` (neu)
beschreibt die dritte Instanz-Form neben „frisch aufsetzen" und „Repo klonen": eine private
Arbeitsinstanz, die Stack-Updates von einem öffentlichen Upstream per `git merge` zieht und
deren eigener Inhalt nie zurückwandert. Der Grund, warum das dem Tarball-Weg vorzuziehen ist,
steht dort ausformuliert — `cp -r` hat keinen Drei-Wege-Merge und keine Konflikterkennung.
Schritt 4 der Anleitung ist das Gate, und zwar ausdrücklich **vor** dem ersten `publish`:
später hinzugefügt schützt es das Fenster nicht, das es schließen soll.
**Dateien:** `config.PUBLISH_REMOTES_FILENAME`, `git_publish.read_allowed_push_urls()`,
`push_url_for()`, `publish_remote_refusal()` und die Prüfung vor dem Reconcile-Schritt,
`doctor.check_publish_remotes()`, `.gitignore`, `instructions/gates.md`,
`instructions/private-instance.md` (neu), `AGENTS.md` (Gate-Liste und Bootstrap-Routing),
12 neue Tests in `test_git_publish.py`.
---
## 2.1.1 - 2026-09-01 - raw_dir-Fixture kappt config.ROOT; letzte private Fixture-Namen ersetzt ## 2.1.1 - 2026-09-01 - raw_dir-Fixture kappt config.ROOT; letzte private Fixture-Namen ersetzt
**Author:** Torben Nehmer **Author:** Torben Nehmer
@@ -47,7 +658,7 @@ Nachgewiesen, indem `raw/documents/` lokal entfernt und die Suite erneut gefahre
Provenance-Tests grün ohne das Verzeichnis, vorher rot. Provenance-Tests grün ohne das Verzeichnis, vorher rot.
**Fixture-Namen.** Die Suite benutzte weiterhin reale Systemnamen der Ursprungsinstanz als **Fixture-Namen.** Die Suite benutzte weiterhin reale Systemnamen der Ursprungsinstanz als
Fixture-Bezeichner (`atlantis`, `abydos`, `Nathan`, `hermes`, `ecodms`, `Opa Hasso`). In einem Fixture-Bezeichner. In einem
öffentlichen Repo beschreiben sie nichts, verraten aber die Namensgebung einer privaten öffentlichen Repo beschreiben sie nichts, verraten aber die Namensgebung einer privaten
Umgebung. Ersetzt durch `aurora`, `almanac`, `Borealis`, `gateway`, `docstore`. `gdeploy` Umgebung. Ersetzt durch `aurora`, `almanac`, `Borealis`, `gateway`, `docstore`. `gdeploy`
bleibt: die Seite existiert im öffentlichen Korpus. bleibt: die Seite existiert im öffentlichen Korpus.
@@ -102,8 +713,7 @@ oder IP-Literale wurde erwogen und verworfen: Der eigene Host steht legitim in `
oder bei jedem Export Fehlalarm geben. oder bei jedem Export Fehlalarm geben.
**Beispiele neutralisiert.** Hilfetexte, Docstrings und Contract-Beispiele nannten reale **Beispiele neutralisiert.** Hilfetexte, Docstrings und Contract-Beispiele nannten reale
Seiten und Pfade der Ursprungsinstanz (Quellenseiten und Pfade Quellenseiten und Pfade der Ursprungsinstanz. In einer
aus dem privaten Korpus). In einer
Distribution ist das nicht nur unpassend, sondern verrät den Inhalt der Instanz, die sie gebaut Distribution ist das nicht nur unpassend, sondern verrät den Inhalt der Instanz, die sie gebaut
hat. Ersetzt durch Beispiele aus dem verbleibenden öffentlichen Korpus. hat. Ersetzt durch Beispiele aus dem verbleibenden öffentlichen Korpus.
+239
View File
@@ -0,0 +1,239 @@
# MCP-Leseserver installieren
Dieses Dokument richtet sich an Menschen. Es beschreibt, wie der MCP-Leseserver eines Chemenu-
Wikis lokal läuft, wie ein Client ihn einbindet, und wie er hinter einer Authentifizierung
erreichbar wird. Der agent-seitige Betriebsablauf steht in
[instructions/mcp-read-server.md](instructions/mcp-read-server.md); die vollständige
Kommandoreferenz in [tools/CONTRACT.md](tools/CONTRACT.md).
**Was der Server ist.** Ein zweiter Konsument desselben Kerns, nicht ein zweites Programm. CLI
und Server rufen dieselben Funktionen auf; ein Golden-Test hält ihre Ausgaben gegeneinander.
Was `tools/wikitool search --json` liefert, liefert das MCP-Tool `search` auch — plus den
Commit, aus dem die Antwort berechnet wurde.
**Was er nicht ist.** Kein Schreibpfad. Es gibt kein Tool, das eine Seite anlegt, ändert oder
publiziert — nicht weil eine Liste gefiltert wird, sondern weil der Server nichts unter
`tools/chemenu/commands/` importiert. Die Funktionen sind aus diesem Prozess heraus nicht
erreichbar.
## Voraussetzungen
- Eine funktionierende Instanz nach [INSTALL.md](INSTALL.md) — inklusive `tools/.venv` und
`ripgrep`
- Python 3.11 oder neuer (wie die CLI)
## Schritt 1: Abhängigkeit installieren
Sie liegt bewusst nicht in `tools/requirements.txt`. Eine Instanz, die nur die CLI benutzt, soll
dafür nicht pydantic, starlette, uvicorn und cryptography mitinstallieren müssen.
```bash
tools/.venv/bin/pip install -r tools/requirements-mcp.txt
```
## Schritt 2: Lokal starten (stdio)
`stdio` ist der Weg zum Ausprobieren und für einen Client auf derselben Maschine: ein Prozess
pro Konsument, lokal gestartet, kein Netzwerk.
```bash
WIKI_TRACE=0 tools/.venv/bin/python -m chemenu.mcp
```
Der Prozess spricht MCP über stdin/stdout und gibt für sich genommen nichts aus — das ist
richtig so. Gestartet wird er normalerweise nicht von Hand, sondern vom Client (Schritt 3).
**`WIKI_TRACE=0` ist nicht optional.** Telemetrie ist per Default an und schreibt nach
`reports/telemetry/` im Repo. Der Server **verweigert den Start**, solange das so ist, statt
still umzuleiten:
```
ERROR Telemetry is on and would write into the served checkout (...). Set WIKI_TRACE=0,
or point WIKI_TRACE_DIR outside the corpus.
```
Beide Auswege sind gleichwertig: `WIKI_TRACE=0` schaltet ab, `WIKI_TRACE_DIR=/var/log/chemenu`
lenkt um. Der Grund steht in Schritt 6 — der Sync darf `reports/` wegräumen.
## Schritt 3: Einen Client einbinden
Die Konfiguration folgt der üblichen MCP-Client-Form. Absolute Pfade, weil der Client kein
Arbeitsverzeichnis erbt:
```json
{
"mcpServers": {
"chemenu": {
"command": "/pfad/zur/instanz/tools/.venv/bin/python",
"args": ["-m", "chemenu.mcp"],
"cwd": "/pfad/zur/instanz/tools",
"env": {
"WIKI_TRACE": "0",
"CHEMENU_ROOT": "/pfad/zur/instanz"
}
}
}
}
```
`CHEMENU_ROOT` sagt, **welches** Wiki bedient wird. Ohne die Variable nimmt der Server den
Checkout, in dem das Paket selbst liegt — für eine einzelne Instanz reicht das, aber wer mehrere
Korpora hat, setzt sie besser immer.
Danach kennt der Client fünf Werkzeuge:
| Tool | Was es beantwortet |
|---|---|
| `search` | Seiten in `kb/` nach Text, nach Frontmatter (`confidence<0.6`, `tags~k8s`) oder beidem |
| `types` | Welche Seitentypen dieses Wiki kennt |
| `describe_type` | Der vollständige Vertrag eines Typs: Felder, Pflichtangaben, Enums |
| `lint` | Strukturelle Befunde: kaputte Wikilinks, Waisen, Index-Drift, Schema-Lücken |
| `status` | Momentaufnahme: Seitenzahl, Verteilung auf Collections, Befundzahlen |
## Schritt 4: Ausgeliefert starten (streamable HTTP)
Der Transport für den Betrieb, und der einzige, vor den sich ein Reverse Proxy setzen kann.
```bash
WIKI_TRACE=0 CHEMENU_ROOT=/srv/chemenu \
tools/.venv/bin/python -m chemenu.mcp \
--transport streamable-http --host 0.0.0.0 --port 8000
```
Der Endpunkt ist dann `http://<host>:8000/mcp`.
`--host 0.0.0.0` ist bewusst nicht der Default. Ohne die Angabe bindet der Server auf Loopback,
was lokal richtig und im Container falsch ist — dort muss der Proxy ihn erreichen können. Wer
`0.0.0.0` setzt, muss also auch dafür sorgen, dass davor etwas steht (Schritt 5).
`sse` wird nicht angeboten. Es ist der abgelöste Remote-Transport; jetzt darauf zu bauen
verschiebt den Wechsel nur.
## Schritt 5: Authentifizierung davor
**Der Server authentifiziert nicht selbst, und das ist Absicht.** Nicht sauber
authentifizierte Zugriffe sollen den Python-Prozess gar nicht erst erreichen. Die
Authentifizierung ist eine Traefik-ForwardAuth-Middleware:
> **<https://gitea.nehmer.net/torben/gitea-mcp-forward-auth>**
Kurz, was sie tut: sie prüft `Authorization: Bearer <token>` gegen SHA-256-Hashes erlaubter
Tokens, antwortet `200` bei gültigem und `401` bei fehlendem oder falschem Token, und hält
`GET /healthz` immer offen. Klartext-Tokens liegen weder in der Konfiguration noch im Log — nur
Hashes und ein kurzer Fingerprint. Konfiguriert wird sie über
`AUTH_PROXY_TOKEN_HASHES_DIR` (ein Verzeichnis, eine Datei je Token-Hash — passend für ein
Kubernetes-Secret-Volume) oder `AUTH_PROXY_TOKEN_HASHES` (kommagetrennte Liste). Einzelheiten,
Referenzmanifeste und ein Testskript stehen im README dort.
Einen Token-Hash erzeugen:
```bash
echo -n "mein-token" | sha256sum | awk '{print $1}'
```
**Rate Limiting gehört an dieselbe Stelle** — vor den Prozess, neben die Authentifizierung.
Nicht in den Iteration Budget Gate: der begrenzt eine *Agenten-Session* am unbemerkten Iterieren
über den Wiki-Zustand, weshalb Retrieval von ihm ausgenommen ist. Ihn als Rate Limiter zu
benutzen würde ihn dazu verwässern.
## Schritt 6: Den Korpus aktuell halten
Der Server liest den Arbeitsbaum. Ein veralteter Checkout antwortet selbstbewusst falsch —
deshalb trägt **jede Antwort den Commit**, aus dem sie berechnet wurde:
```json
{ "commit": "<40-stelliger SHA>", "as_of": "<ISO-8601, UTC>", "count": 3, "...": "..." }
```
Aktuell gehalten wird der Baum durch Polling, aus einem Timer neben dem Server:
```bash
git -C "$CHEMENU_ROOT" fetch --quiet origin && \
git -C "$CHEMENU_ROOT" reset --hard --quiet origin/main
```
Polling statt Webhook, weil es keinen eingehenden Endpunkt und keine Signaturprüfung braucht —
eine kleinere Angriffsfläche als das, was es optimieren würde.
`reset --hard` ist dabei tragend und keine Bequemlichkeit: der Korpus-Cache verwendet einen
Parse wieder, solange der Commit gleich bleibt, und cacht einen **schmutzigen Baum überhaupt
nicht**. Ein abgedrifteter Checkout antwortet also zwar richtig, parst aber bei jeder Anfrage
neu — und stempelt jede Antwort mit `"commit": null`, weil sie keiner Revision entspricht.
## Verifikation
Läuft es? Der schnellste Test ohne Client — startet den Server über stdio, listet die Tools und
stellt eine Frage:
```bash
cd tools && WIKI_TRACE=0 .venv/bin/python - <<'EOF'
import asyncio, os
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
async def main():
params = StdioServerParameters(
command=".venv/bin/python", args=["-m", "chemenu.mcp"],
env={"WIKI_TRACE": "0", "PATH": os.environ["PATH"]},
)
async with stdio_client(params) as (r, w):
async with ClientSession(r, w) as s:
await s.initialize()
print("Tools:", [t.name for t in (await s.list_tools()).tools])
out = await s.call_tool("status", {})
d = getattr(out, "structuredContent", None) or out.structured_content
print("Seiten:", d["pages"], "| Commit:", d["commit"])
asyncio.run(main())
EOF
```
Erwartete Ausgabe, sinngemäß:
```
Tools: ['search', 'types', 'describe_type', 'lint', 'status']
Seiten: 176 | Commit: 576df2cdddc96614a7e6641e562022d52112d411
```
Gegen die CLI gegenprüfen — beide müssen dieselbe Antwort geben:
```bash
tools/wikitool search "<begriff>" --json
```
## Troubleshooting
**`ERROR Telemetry is on and would write into the served checkout`** — erwartetes Verhalten,
kein Fehler in der Installation. `WIKI_TRACE=0` setzen oder `WIKI_TRACE_DIR` aus dem Korpus
heraus zeigen lassen (Schritt 2).
**`ModuleNotFoundError: No module named 'mcp'`** — Schritt 1 fehlt, oder der Client startet ein
anderes Python als das der Instanz. Im Client den absoluten Pfad auf `tools/.venv/bin/python`
setzen.
**`"commit": null` in jeder Antwort** — der bediente Baum hat uncommittete Änderungen. Entweder
läuft der Sync nicht, oder etwas schreibt in den Korpus, das dort nichts zu suchen hat. Der
Server selbst schreibt nie; ein Test prüft das, indem er alle fünf Tools aufruft und Dateibaum,
`HEAD` und `git status --porcelain` vorher/nachher vergleicht.
**`commit` nennt eine alte Revision** — der Sync aus Schritt 6 läuft nicht.
**Der Server antwortet anders als `wikitool`** — das ist ein Defekt, keine
Konfigurationsdifferenz: beide gehen durch dieselben Funktionen, und ein Golden-Test hält sie
zusammen. Zuerst prüfen, ob beide auf denselben Root zeigen; `CHEMENU_ROOT` ist leicht für einen
von beiden gesetzt und für den anderen nicht.
**Von außen nicht erreichbar** — ohne `--host 0.0.0.0` bindet der Server auf Loopback
(Schritt 4). Wenn er dann erreichbar ist, aber jeder Aufruf `401` bekommt, arbeitet die
Middleware aus Schritt 5 korrekt und das Token stimmt nicht.
## Was hier bewusst nicht steht
Deployment — Cluster, Ingress-Hosts, Secret-Store, FluxCD-Quelle. Das ist private Infrastruktur
und dieses Repo ist öffentlich.
Ein **Container-Image** für den Betrieb gibt es noch nicht; es ist als eigenes Vorhaben erfasst,
mitsamt den Entscheidungen, die dafür noch offen sind (Korpus im Image oder als Volume, wer den
Sync ausführt, Basis-Image, Healthcheck):
<https://gitea.nehmer.net/torben/chemenu/issues/37>. Bis dahin ist der Weg oben — venv,
`python -m chemenu.mcp`, Proxy davor — der vollständige.
+91 -35
View File
@@ -1,10 +1,15 @@
# Installation # Installation
Dieses Dokument richtet sich an Menschen. Es gibt drei Wege: ein **Release herunterladen** Dieses Dokument richtet sich an Menschen. Es gibt vier Wege: ein **Release herunterladen**
(der normale Weg zu einer neuen Instanz), eine Distribution **selbst exportieren**, oder (der normale Weg zu einer neuen Instanz), eine Distribution **selbst exportieren**, **dieses
**dieses Repo klonen** (Torbens persönliche Wiki, samt Inhalt). Der agent-seitige Ablauf steckt Repo klonen** (Testbett und Demo, samt Beispielkorpus), oder eine **private Instanz mit diesem
in `instructions/`; hier stehen nur die menschlichen Teile - für die vollständige Repo als Upstream** aufsetzen. Der agent-seitige Ablauf steckt in `instructions/`; hier stehen
Kommandoreferenz siehe [tools/CONTRACT.md](tools/CONTRACT.md). nur die menschlichen Teile - für die vollständige Kommandoreferenz siehe
[tools/CONTRACT.md](tools/CONTRACT.md).
Den optionalen **MCP-Leseserver** installiert und betreibt
[INSTALL-MCP.md](INSTALL-MCP.md): derselbe Korpus, lesend, für einen Konsumenten, der kein
Terminal auf dieser Maschine ist.
## Voraussetzungen ## Voraussetzungen
@@ -16,19 +21,22 @@ Kommandoreferenz siehe [tools/CONTRACT.md](tools/CONTRACT.md).
## Weg A: Release herunterladen ## Weg A: Release herunterladen
Der kürzeste Weg zu einer eigenen Instanz - kein Checkout dieses Repos nötig. Jedes Release Der kürzeste Weg zu einer eigenen Instanz - kein Checkout dieses Repos nötig. Jedes Release
trägt genau einen `dist export`-Baum plus eine Prüfsumme. Das Repo ist derzeit privat, der trägt genau einen `dist export`-Baum plus eine Prüfsumme. Das Repo ist öffentlich, der Download
Download braucht also ein Gitea-Token mit Lesezugriff (siehe braucht also weder Konto noch Token:
[Konfiguration](#konfiguration)):
```bash ```bash
BASE=https://gitea.nehmer.net/torben/chemenu/releases/download/v<version> BASE=https://gitea.nehmer.net/torben/chemenu/releases/download/v<version>
curl -LO -H "Authorization: token $WIKITOOL_UPDATE_TOKEN" $BASE/chemenu-stack-<version>.tar.gz curl -LO $BASE/chemenu-stack-<version>.tar.gz
curl -LO -H "Authorization: token $WIKITOOL_UPDATE_TOKEN" $BASE/chemenu-stack-<version>.tar.gz.sha256 curl -LO $BASE/chemenu-stack-<version>.tar.gz.sha256
sha256sum -c chemenu-stack-<version>.tar.gz.sha256 sha256sum -c chemenu-stack-<version>.tar.gz.sha256
tar xzf chemenu-stack-<version>.tar.gz tar xzf chemenu-stack-<version>.tar.gz
cd chemenu-stack-<version> cd chemenu-stack-<version>
``` ```
Die Prüfsumme ist nicht Zierde: Sie ist das Einzige, was einen unterbrochenen Download von
einem vollständigen unterscheidet, und `sha256sum -c` muss `OK` sagen, bevor irgendetwas
entpackt wird.
Danach weiter mit Schritt 2 aus Weg B: den Agenten Danach weiter mit Schritt 2 aus Weg B: den Agenten
[instructions/setup-instance.md](instructions/setup-instance.md) ausführen lassen. Der [instructions/setup-instance.md](instructions/setup-instance.md) ausführen lassen. Der
entpackte Baum ist bereits eine Distribution - Schritt 1 (`dist export`) entfällt. entpackte Baum ist bereits eine Distribution - Schritt 1 (`dist export`) entfällt.
@@ -60,11 +68,14 @@ Zwei Schritte, von denen nur der erste rein menschlich ist:
Wiki-Seite (`$WIKI_AUTHOR` überschreibt dies bei Bedarf). Wiki-Seite (`$WIKI_AUTHOR` überschreibt dies bei Bedarf).
- **Remote** (optional) - eine URL, wenn du das Repo auf einen Server pushen willst; sonst - **Remote** (optional) - eine URL, wenn du das Repo auf einen Server pushen willst; sonst
bleibt die Instanz lokal, und jedes `publish` läuft mit `--no-push`. bleibt die Instanz lokal, und jedes `publish` läuft mit `--no-push`.
- **KB-Sprache** - die exportierte Distribution bringt **Deutsch** mit: die Regel in - **Autorenkonventionen** - Sprache, Abschnittsnamen, Namensformen, Ton, Beziehungslabels
`kb/CONTRACT.md`, das Vokabular in `instructions/german-terminology.md` und deutsche und Confidence-Rubrik stehen in `kb/CONVENTIONS.md`, dazu je Collection die Regeln in
Abschnittsnamen in den Seitenvorlagen. Das ist eine Entscheidung dieser Ursprungsinstanz, `kb/<name>/COLLECTION.md`. Die Distribution bringt davon nur die `.template`-Dateien mit:
keine Eigenschaft des Musters. Willst du eine andere Sprache, sag es **vor dem ersten das sind Entscheidungen *dieser* Instanz, keine Eigenschaft des Musters, und nichts davon
Ingest** - danach ist es eine Migration jeder bereits angelegten Seite. liegt unter `tools/` oder `types/`. Fertige Profile - darunter ein vollständiges deutsches -
hält `instructions/kb-profiles.md` bereit; es ist eine Palette, kein Enum. Sag die Sprache
**vor dem ersten Ingest** - danach ist ein Wechsel der Abschnittsnamen eine Migration jeder
bereits angelegten Seite.
- **Personalization** - wer diese Instanz bedient (`USER.md`) und wie sie klingt - **Personalization** - wer diese Instanz bedient (`USER.md`) und wie sie klingt
(`SOUL.md`). Die Distribution bringt nur `USER.md.template` und `SOUL.md.template` mit: (`SOUL.md`). Die Distribution bringt nur `USER.md.template` und `SOUL.md.template` mit:
persönlicher Inhalt gehört nicht in jede exportierte Kopie, aber beide Dateien werden in persönlicher Inhalt gehört nicht in jede exportierte Kopie, aber beide Dateien werden in
@@ -82,10 +93,14 @@ Zwei Schritte, von denen nur der erste rein menschlich ist:
## Weg C: Dieses Repo klonen ## Weg C: Dieses Repo klonen
Für Torbens Instanz selbst, oder einen Fork davon samt Inhalt: Für die Arbeit am Stack selbst, oder um sich den mitgelieferten Korpus als begehbares Beispiel
anzusehen. Was hier liegt, ist ein **Testbett und eine Demo**, keine produktive Wissensbasis:
rund 170 Seiten, die den Stack selbst dokumentieren - Gates, Lint, Versionierung, Suche, das
Wiki-Muster. Wer eigenes Wissen sammeln will, nimmt Weg A oder B und fängt mit einem leeren
`kb/` an.
```bash ```bash
git clone <repo-url> git clone https://gitea.nehmer.net/torben/chemenu.git
cd chemenu cd chemenu
``` ```
@@ -102,6 +117,21 @@ Checkout* beschreibt und nicht das Repo. Sie ist optional; wer sie anlegt, spart
folgenden Session die Fragen nach Harness, MCP-Servern und Remote. Vorlage: folgenden Session die Fragen nach Harness, MCP-Servern und Remote. Vorlage:
`ENVIRONMENT.md.template`, Ablauf: Schritt 5 in `instructions/bootstrap.md`. `ENVIRONMENT.md.template`, Ablauf: Schritt 5 in `instructions/bootstrap.md`.
## Weg D: Private Instanz mit diesem Repo als Upstream
Die Kombination aus A und C: eine eigene, nicht öffentliche Instanz, die weiterhin
Stack-Updates von hier zieht - per `git merge` statt per Tarball, also mit echtem
Drei-Wege-Merge statt `cp -r`.
Das ist der Weg mit dem höchsten Einsatz, weil ein Checkout dann zwei Remotes hat und git beim
Push nicht unterscheidet, welcher welcher ist. Ein falsches `--remote` legt privaten Inhalt auf
ein öffentliches Repo, und ein Force-Push holt das nicht zurück - die Objekte bleiben per SHA
abrufbar, bis auf dem Server die Reflogs verfallen.
Dagegen gibt es das **Publish-Remote-Gate**, und die Anleitung setzt es an die Stelle, an der
es wirkt: *vor* dem ersten `publish`. Vollständiges Vorgehen:
[instructions/private-instance.md](instructions/private-instance.md).
## Version und Updates ## Version und Updates
Jede Instanz trägt die Version des **Stacks** (Werkzeuge, Typen, Instruktionen, Contracts) - Jede Instanz trägt die Version des **Stacks** (Werkzeuge, Typen, Instruktionen, Contracts) -
@@ -119,8 +149,21 @@ nicht erreichbarer Feed wird als Fehler gemeldet - **nie** als „aktuell".
**Was die Versionsnummer aussagt:** kompatibel ist, was in der *linkesten von Null **Was die Versionsnummer aussagt:** kompatibel ist, was in der *linkesten von Null
verschiedenen Stelle* übereinstimmt. `0.1.3 → 0.1.4` ist ein sicheres Update, `0.1.3 → 0.2.0` verschiedenen Stelle* übereinstimmt. `0.1.3 → 0.1.4` ist ein sicheres Update, `0.1.3 → 0.2.0`
verlangt eine Migration, und ab `1.0.0` liest sich dieselbe Regel als das gewohnte „MAJOR heißt nicht, und ab `1.0.0` liest sich dieselbe Regel als das gewohnte „MAJOR bricht". `version check`
Migration". `version check` sagt das direkt (`state: update` vs. `state: migration`). sagt das direkt (`state: update` vs. `state: migration`).
Was diese Stelle beantwortet, ist **ob die neue Version ein Drop-in-Ersatz ist** - ob sich die
Maschinerie einfach darüberkopieren lässt und ob die alte danach noch zurückkann. Ob *Inhalt*
migriert werden muss, ist eine **zweite, unabhängige Frage**. Ein MAJOR-Sprung kann eine leere
Migrationskette haben und trotzdem Handarbeit verlangen: umbenannter Release-Feed, umbenanntes
Artefakt, umbenannter Import- oder Kommandoname, geänderte Envvar - `kb/` bleibt dabei
unangetastet, das Update ist trotzdem keins zum Drüberkopieren. Der Abschnitt „Sonderfall:
Update von 1.x auf 2.0.0" unten ist genau dieser Fall.
Deshalb stehen in den Release-Notes eines MAJOR zwei getrennte Zeilen, und beide sind vor dem
Update zu lesen: **Breaking Change:** sagt, was aufhört zu funktionieren und was diese Instanz
dagegen tun muss; **Migration:** sagt, ob und wie der Korpus umgeschrieben wird (`none required`,
wenn nicht). `tools/wikitool version notes` druckt den Eintrag.
### Eine Instanz aktualisieren ### Eine Instanz aktualisieren
@@ -137,13 +180,17 @@ dieser Unterschied ist der Zustand, in dem sich jede Instanz mitten im Upgrade b
2. Release-Tarball herunterladen und entpacken (Weg A), die Release-Notes lesen. 2. Release-Tarball herunterladen und entpacken (Weg A), die Release-Notes lesen.
3. Die **Maschinerie** aus dem Tarball über die Instanz kopieren: `tools/`, `types/`, 3. Die **Maschinerie** aus dem Tarball über die Instanz kopieren: `tools/`, `types/`,
`instructions/`, `AGENTS.md`, `VERSION`, `.wikitool-release.json`. Nicht anfassen: `kb/`, `instructions/`, `AGENTS.md`, `VERSION`, `.wikitool-release.json` - **und `kb/CONTRACT.md`**.
`raw/`, `work/`, `.wikitool-kb.json` und `.git/` - das ist die Instanz selbst. Die letzte Datei liegt unter einem Content-Verzeichnis, ist aber Stack-Eigentum: sie hält,
4. Achtung bei lokal angepassten Contract-Dateien: wer z. B. die KB-Sprache umgestellt hat was `wikitool` erzwingt, und ist in jeder Instanz gleich. Nicht anfassen: alles andere unter
(Schritt 5 in `setup-instance.md`), hat `kb/CONTRACT.md` und die Templates unter `types/` `kb/` und `raw/`, `work/`, `.wikitool-kb.json` und `.git/` - das ist die Instanz selbst,
verändert. Diese Änderungen vorher sichern und danach wieder einspielen. Welche Dateien das `kb/CONVENTIONS.md` und die `kb/*/COLLECTION.md` eingeschlossen.
sind, verrät ein Vergleich gegen die sha256-Summen im `files`-Block der alten 4. Achtung bei lokal angepassten Stack-Dateien. Die Autorenkonventionen gehören **nicht** dazu:
`.wikitool-release.json`. `kb/CONVENTIONS.md` und die `kb/*/COLLECTION.md` liegen unter `kb/`, werden in Schritt 3
also ohnehin nicht angefasst - genau dafür ist der Schnitt da. Wer darüber hinaus etwas
unter `tools/`, `types/` oder `instructions/` verändert hat, sichert das vorher und spielt
es danach wieder ein. Welche Dateien das sind, verrät ein Vergleich gegen die sha256-Summen
im `files`-Block der alten `.wikitool-release.json`.
5. **Die Migrationskette abarbeiten.** `tools/wikitool migrate status` listet jetzt alle 5. **Die Migrationskette abarbeiten.** `tools/wikitool migrate status` listet jetzt alle
offenen Migrationen in der Reihenfolge, in der sie laufen müssen - bei einem Sprung über offenen Migrationen in der Reihenfolge, in der sie laufen müssen - bei einem Sprung über
mehrere Versionen sind das mehrere. Für jede: das genannte Dokument unter mehrere Versionen sind das mehrere. Für jede: das genannte Dokument unter
@@ -199,22 +246,25 @@ behält Schema und Shape.
| `WIKI_AUTHOR` | Override für den Autornamen neuer Source-Seiten | `git config user.name` - fehlt beides, bricht `new` mit `ERROR` ab | | `WIKI_AUTHOR` | Override für den Autornamen neuer Source-Seiten | `git config user.name` - fehlt beides, bricht `new` mit `ERROR` ab |
| `WIKITOOL_SESSION_ID` | Scopt das Iteration-Budget-Gate auf eine Aufgabe statt auf ein Terminal | Parent-Process-ID (siehe [instructions/session-setup.md](instructions/session-setup.md)) | | `WIKITOOL_SESSION_ID` | Scopt das Iteration-Budget-Gate auf eine Aufgabe statt auf ein Terminal | Parent-Process-ID (siehe [instructions/session-setup.md](instructions/session-setup.md)) |
| `WIKITOOL_UPDATE_URL` | Release-Feed, den `version check` abfragt | Wert aus `.wikitool-release.json`, sonst der Feed der Ursprungs-Instanz | | `WIKITOOL_UPDATE_URL` | Release-Feed, den `version check` abfragt | Wert aus `.wikitool-release.json`, sonst der Feed der Ursprungs-Instanz |
| `WIKITOOL_UPDATE_TOKEN` | Gitea-Token für den Release-Feed | keiner - **aber das Ursprungs-Repo ist derzeit privat, also wird ein Token gebraucht** (siehe unten) | | `WIKITOOL_UPDATE_TOKEN` | Gitea-Token für den Release-Feed | keiner - gegen `torben/chemenu` nicht nötig, nur für einen privaten Fork (siehe unten) |
| `CHEMENU_ROOT` | Auf welchen Korpus das Paket zeigt - für einen Aufrufer, der nicht im Checkout selbst liegt | der Checkout, in dem das Paket liegt (`tools/wikitool` verhält sich ohne die Variable unverändert) |
| `WIKI_TRACE` / `WIKI_TRACE_DIR` | Telemetrie abschalten bzw. aus dem Arbeitsbaum heraus umlenken | an, nach `reports/telemetry/` - der MCP-Server verweigert damit den Start, siehe [INSTALL-MCP.md](INSTALL-MCP.md) |
**Privates Ursprungs-Repo.** `torben/chemenu` ist nicht öffentlich lesbar. Gitea **Gegen das Ursprungs-Repo braucht es kein Token.** `torben/chemenu` ist öffentlich lesbar;
antwortet anonymen Aufrufern für ein unsichtbares Repo mit demselben `404` wie für ein gar `version check` und der Download in Weg A funktionieren ohne Konfiguration.
nicht existierendes - ein fehlendes Release und ein fehlender Zugriff sehen also identisch aus.
Für `version check` (und für den Download in Weg A) braucht es deshalb ein Gitea-Token mit **Für einen privaten Fork schon.** Wer den Stack in ein eigenes, nicht öffentliches Repo legt
Lesezugriff: und `WIKITOOL_UPDATE_URL` auf dessen Feed zeigen lässt, stößt auf eine Eigenheit, die man
kennen sollte: Gitea antwortet anonymen Aufrufern für ein unsichtbares Repo mit demselben
`404` wie für ein gar nicht existierendes. Ein fehlendes Release und ein fehlender Zugriff
sehen dann identisch aus - „kein Update gefunden" wäre in dem Fall schlicht gelogen. Dagegen
hilft ein Gitea-Token mit Lesezugriff:
```bash ```bash
export WIKITOOL_UPDATE_TOKEN="<gitea-token>" export WIKITOOL_UPDATE_TOKEN="<gitea-token>"
tools/wikitool version check tools/wikitool version check
``` ```
Wird das Repo öffentlich geschaltet, entfällt das Token ersatzlos - der Feed ist dann anonym
lesbar und `version check` funktioniert ohne Konfiguration.
## Verifikation ## Verifikation
```bash ```bash
@@ -259,6 +309,12 @@ tools/wikitool instructions verify
sondern die Aufforderung, die Ausgabe einem Menschen zu zeigen: sie enthält die vollständige sondern die Aufforderung, die Ausgabe einem Menschen zu zeigen: sie enthält die vollständige
Dateiliste und die exakte `--confirm <token>`-Zeile, die nach Freigabe veröffentlicht. Dateiliste und die exakte `--confirm <token>`-Zeile, die nach Freigabe veröffentlicht.
Details: [instructions/gates.md](instructions/gates.md). Details: [instructions/gates.md](instructions/gates.md).
- **`publish` endet mit Exit-Code 42 (Publish-Remote-Gate)** - dieser Checkout hat eine
`.wikitool-remotes.json`, und das angesteuerte Remote steht nicht darin. Ebenfalls kein
Fehler: Die Ausgabe nennt die Push-URL, an die geschrieben würde, und die erlaubten. Anders
als beim Mass-Update-Gate gibt es hier **keinen Token und keine Flagge** - stimmt das Ziel
wirklich, trägt der Mensch dessen URL selbst in die Datei ein. Ein Agent, der die Datei
anfasst, um an der Verweigerung vorbeizukommen, öffnet ein Gate aus eigenem Antrieb.
- **Ich will am Tool-Stack selbst weiterarbeiten (nicht nur Wiki-Inhalt betreiben)** - eine neue - **Ich will am Tool-Stack selbst weiterarbeiten (nicht nur Wiki-Inhalt betreiben)** - eine neue
Instanz hat dafür keinen Weg: `dist export` lässt `instructions/dev/` (Stack-Entwicklung, Instanz hat dafür keinen Weg: `dist export` lässt `instructions/dev/` (Stack-Entwicklung,
inkl. der vendorten `commonplace/`-Wissensbasis) bewusst und dauerhaft weg, ohne inkl. der vendorten `commonplace/`-Wissensbasis) bewusst und dauerhaft weg, ohne
+37 -5
View File
@@ -14,12 +14,15 @@ and maintains a persistent wiki** that compounds over time.
English; the compiled pages under `kb/` are not. What stays English inside them is everything that English; the compiled pages under `kb/` are not. What stays English inside them is everything that
is an *identifier* rather than prose - page titles, section headings, wikilink targets, citation is an *identifier* rather than prose - page titles, section headings, wikilink targets, citation
ids, schema enum values, tags, commands, paths and code - so `GitOps Ownership Model` and ids, schema enum values, tags, commands, paths and code - so `GitOps Ownership Model` and
`## Beziehungen` sit in the same page without contradiction. The rule is `## Beziehungen` sit in the same page without contradiction. Which lines are identifiers is
[kb/CONTRACT.md § Language](kb/CONTRACT.md#language); the vocabulary behind it is [kb/CONTRACT.md § Language and identifiers](kb/CONTRACT.md#language-and-identifiers); *which
language* the prose is in, and what the tool-owned headings are called, is this instance's own
[kb/CONVENTIONS.md](kb/CONVENTIONS.md), and the vocabulary behind it is
[instructions/german-terminology.md](instructions/german-terminology.md). [instructions/german-terminology.md](instructions/german-terminology.md).
This is a per-instance decision, not a property of the pattern. A new instance built with This is a per-instance decision, not a property of the pattern - which is why it lives in a file
`dist export` starts empty and can pick any language by editing that one contract section before the instance owns rather than in one the stack ships. A new instance built with
`dist export` starts empty and picks any language by filling in `kb/CONVENTIONS.md` before
the first ingest. the first ingest.
## Getting started ## Getting started
@@ -51,6 +54,7 @@ chemenu/
├── CLAUDE.md # Claude Code only: imports AGENTS.md/USER.md/SOUL.md/ENVIRONMENT.md + its Claude-Code-only instructions. No rules of its own ├── CLAUDE.md # Claude Code only: imports AGENTS.md/USER.md/SOUL.md/ENVIRONMENT.md + its Claude-Code-only instructions. No rules of its own
├── README.md # This file: human-readable overview of the whole repo ├── README.md # This file: human-readable overview of the whole repo
├── INSTALL.md # Human-readable setup: new instance vs. cloning this one ├── INSTALL.md # Human-readable setup: new instance vs. cloning this one
├── INSTALL-MCP.md # Human-readable setup for the optional MCP read server
├── EVALS.md # Human-readable overview of telemetry and evaluation ├── EVALS.md # Human-readable overview of telemetry and evaluation
├── CHANGES.md # Changelog for the stack itself ├── CHANGES.md # Changelog for the stack itself
├── USER.md # Who operates this instance - context, never instructions ├── USER.md # Who operates this instance - context, never instructions
@@ -342,6 +346,34 @@ under `instructions/dev/` (never present in a distributed instance - `tools/CONT
explains why). explains why).
<!-- dist:strip-end --> <!-- dist:strip-end -->
### MCP read server (optional)
The terminal is not the only way in. `tools/chemenu/mcp/` serves the same wiki read-only over
MCP - `search`, `types`, `describe_type`, `lint` and `status` - so a consumer that is not a
shell on this machine can ask the same questions and get the same answers. Literally the same:
the CLI and the server are two adapters over one core, and a golden test holds their output
together rather than trusting that it agrees.
There is no tool that writes, and not because one is filtered out of a list: the server imports
nothing under `chemenu/commands/`, so `new`, `publish` and the rest are unreachable from it.
Every answer carries the commit it was computed from, so a checkout that has fallen behind
produces a visibly stale answer instead of a confident wrong one.
```bash
tools/.venv/bin/pip install -r tools/requirements-mcp.txt
WIKI_TRACE=0 tools/.venv/bin/python -m chemenu.mcp # stdio
WIKI_TRACE=0 tools/.venv/bin/python -m chemenu.mcp \
--transport streamable-http --host 0.0.0.0 --port 8000 # deployed
```
The dependency is deliberately not in `requirements.txt`: an instance that only uses the CLI
should not have to install a web stack to do it.
Installing it, wiring a client to it, and putting authentication in front of it:
[`INSTALL-MCP.md`](INSTALL-MCP.md), for humans. The agent-side operating procedure - keeping the
checkout current, and what a stale answer looks like - is
[`instructions/mcp-read-server.md`](instructions/mcp-read-server.md).
### Obsidian ### Obsidian
Open this directory in Obsidian for: Open this directory in Obsidian for:
@@ -372,7 +404,7 @@ This wiki is tailored for IT work with:
- **Entity types** specific to software development and systems - **Entity types** specific to software development and systems
- **Relationship types** like `hängt ab von`, `verwendet`, `implementiert` - the vocabulary is in - **Relationship types** like `hängt ab von`, `verwendet`, `implementiert` - the vocabulary is in
[kb/CONTRACT.md § Linking](kb/CONTRACT.md#linking) [kb/CONVENTIONS.md](kb/CONVENTIONS.md), because it is this instance's rather than the stack's
- **Templates** for projects, systems, tools, technologies, ADRs - **Templates** for projects, systems, tools, technologies, ADRs
- **Guidelines** for documenting technical decisions - **Guidelines** for documenting technical decisions
- **Cross-reference patterns** for code and architecture - **Cross-reference patterns** for code and architecture
+1 -1
View File
@@ -1 +1 @@
2.1.1 4.0.0
+28 -5
View File
@@ -58,16 +58,38 @@ whether an instruction is still reachable, which is exactly why the answer means
`instructions/dev/` boundary check below asks the opposite question - what would *dangle* in a `instructions/dev/` boundary check below asks the opposite question - what would *dangle* in a
distributed instance - and does scan README.md, because `dist export` ships it verbatim. distributed instance - and does scan README.md, because `dist export` ships it verbatim.
Two kinds of file use the Manual tier today: [german-terminology.md](german-terminology.md), a Three kinds of file use the Manual tier today: [german-terminology.md](german-terminology.md), a
vocabulary consulted on demand rather than a procedure, and every migration document (below). vocabulary consulted on demand rather than a procedure; [kb-profiles.md](kb-profiles.md), the
catalogue of authoring profiles an instance may adopt into its own `kb/CONVENTIONS.md` and
`COLLECTION.md` files; and every migration document (below).
## `instructions/migrations/` ## `instructions/migrations/`
A content migration is a Manual instruction with two extra frontmatter fields A content migration is a Manual instruction with three extra frontmatter fields
(`types/instruction.schema.yaml`): `migrates_to:`, the stack version whose content shape it (`types/instruction.schema.yaml`): `migrates_to:`, the stack version whose content shape it
produces, and `migration_kind:` (`mechanical` | `assisted`). It lives at produces; `migration_kind:` (`mechanical` | `assisted`); and `obligation:`
(`required` | `offered`, default `required`). It lives at
`instructions/migrations/<version>-<slug>.md`. `instructions/migrations/<version>-<slug>.md`.
`migration_kind:` and `obligation:` are **two axes, not one**. The first says how the work is
carried out, the second whether it has to happen at all:
| `obligation:` | Means | `migrate status` |
|---|---|---|
| `required` | The content must reach the new shape or it no longer fits the machinery | Counted as outstanding; `migrate done` advances `kb_version` through it, in chain order |
| `offered` | A file the instance owns still works as it is, and the stack proposes a better default | Listed separately, never blocks, no ordering rule. `migrate done` records it in the applied ledger and leaves `kb_version` where it is |
Keeping them apart is what stops `migrate status` crying wolf: an instance nagged about an
improvement it declined stops reading the nag that means its content no longer fits its
machinery. And because taking an offer deliberately does not move the version, the **applied
ledger** - not `kb_version` - is what makes an offer stop being offered; without that record
there is no way to tell a taken offer from an ignored one.
An `offered` migration is what makes an instance-owned file upgradeable at all. `dist export`
records a sha256 per shipped file in `.wikitool-release.json`, so `migrate status` can say which
of those files the instance edited and which it merely received - the first have to be
reconciled by a person, the second can simply be copied over.
The tier fits exactly: a migration must never be picked up implicitly - it rewrites the corpus - The tier fits exactly: a migration must never be picked up implicitly - it rewrites the corpus -
and it is referenced by nothing, because `tools/wikitool migrate status` finds it by reading the and it is referenced by nothing, because `tools/wikitool migrate status` finds it by reading the
directory and comparing `migrates_to:` against this instance's `kb_version`. That is also why directory and comparing `migrates_to:` against this instance's `kb_version`. That is also why
@@ -153,7 +175,8 @@ What lives where:
|-------|------| |-------|------|
| [AGENTS.md](../AGENTS.md) | Invariants and routing - what must always hold | | [AGENTS.md](../AGENTS.md) | Invariants and routing - what must always hold |
| `instructions/` | How the tooling is *operated* | | `instructions/` | How the tooling is *operated* |
| [kb/CONTRACT.md](../kb/CONTRACT.md) + each `COLLECTION.md` | How a page is *authored* | | [kb/CONTRACT.md](../kb/CONTRACT.md) | What the stack enforces about a page, in every instance |
| `kb/CONVENTIONS.md` + each `COLLECTION.md` | What *this* instance decided about authoring - owned by the instance, shipped only as a `.template` |
| [types/](../types/type-spec.md) | What a page structurally *is* | | [types/](../types/type-spec.md) | What a page structurally *is* |
| [tools/CONTRACT.md](../tools/CONTRACT.md) | What each command does and how it fails | | [tools/CONTRACT.md](../tools/CONTRACT.md) | What each command does and how it fails |
+25 -7
View File
@@ -48,6 +48,9 @@ stack development happens in the origin repo instead (see AGENTS.md's routing li
[testing-conventions.md](../testing-conventions.md) - the suite runs against a deliberately [testing-conventions.md](../testing-conventions.md) - the suite runs against a deliberately
empty machine; what the autouse fixture already neutralizes, and what a test still has to empty machine; what the autouse fixture already neutralizes, and what a test still has to
establish itself. Read it before adding or changing a test. establish itself. Read it before adding or changing a test.
[version-parts.md](../version-parts.md) - which part a change bumps: the drop-in test, the
catalogue of breaks that cross the compatibility boundary with `kb/` untouched, and what to
put in front of the user before a breaking bump. Read it before step 3.
More instructions are added here incrementally as stack-development needs come up - this More instructions are added here incrementally as stack-development needs come up - this
list grows without needing this skill file to change shape. list grows without needing this skill file to change shape.
3. **Raise the version, if the change ships.** A change under `tools/`, `types/`, 3. **Raise the version, if the change ships.** A change under `tools/`, `types/`,
@@ -59,18 +62,27 @@ stack development happens in the origin repo instead (see AGENTS.md's routing li
``` ```
Never edit `VERSION` or the entry's heading by hand - `bump` writes both, and `docs verify` Never edit `VERSION` or the entry's heading by hand - `bump` writes both, and `docs verify`
fails a tree where they disagree. Pick the part by what an existing instance would have to do: fails a tree where they disagree. Pick the part by whether the new version is a **drop-in
replacement** for the old one - not by whether content has to be migrated:
| Change | Part | | Change | Part |
|--------|------| |--------|------|
| Fix, no interface change | `--patch` | | Fix, no interface change | `--patch` |
| New capability, backwards compatible | `--minor` | | New capability, still drop-in in both directions | `--minor` |
| **Existing content must be migrated** | `--major` | | **Not a drop-in replacement** - any hand-work by the user or a migration script, or a downgrade that no longer works | `--major` |
A `--major` bump additionally needs a migration document for the new version - written per Content migration is one way to land in the last row, not the definition of it: a rename of
[migrate-corpus.md](../../migrate-corpus.md) - or `--no-migration "<reason>"` when no content the update path, the artefact, an import name, a flag or an envvar breaks a swap with `kb/`
actually has to change. `bump` refuses otherwise, and so does `docs verify`: an instance entirely untouched. The full test, the catalogue of such breaks, and what to put in front of
learning that it must migrate, with nothing telling it how, is a dead end. the user first are in [version-parts.md](../version-parts.md) - **read it before choosing
`--major`.**
A `--major` bump therefore needs two things recorded. `--breaking "<what stops working>"`
is required on every boundary-crossing bump; on top of it, a migration document for the new
version - written per [migrate-corpus.md](../../migrate-corpus.md) - or
`--no-migration "<reason>"` when no content actually has to change. `bump` refuses without
either, and so does `docs verify`: an instance learning that it must migrate, with nothing
telling it how, is a dead end.
Then write the entry's body - `bump` deliberately leaves it empty, the same way `new` leaves Then write the entry's body - `bump` deliberately leaves it empty, the same way `new` leaves
the prose. the prose.
@@ -90,6 +102,12 @@ stack development happens in the origin repo instead (see AGENTS.md's routing li
- **Touches both stack code and wiki content in one session?** Apply this skill's rules to the - **Touches both stack code and wiki content in one session?** Apply this skill's rules to the
code changes and the normal content skills' rules to the content changes - they are not code changes and the normal content skills' rules to the content changes - they are not
mutually exclusive within a session, only per change. mutually exclusive within a session, only per change.
- **The change turns out not to be a drop-in replacement?** Do not bump across the boundary on
your own initiative. Every existing instance pays for a breaking change once, by hand, so the
user decides whether it is worth that: show them what breaks, what an instance has to do about
it, and the alternatives (avoid the break with a shim, defer and batch it with the next one,
or split it behind a deprecation window), then recommend one and wait for a go-ahead.
[version-parts.md](../version-parts.md) step 4 has the full shape.
## Scope ## Scope
+5
View File
@@ -35,6 +35,11 @@ Do not re-do any of this per test; it is done for you, per test, via `monkeypatc
fixture redirects it into `tmp_path`. Tracing is never disabled suite-wide, because two fixture redirects it into `tmp_path`. Tracing is never disabled suite-wide, because two
telemetry tests assert that a trace gets written. telemetry tests assert that a trace gets written.
Two in-process caches are cleared alongside the environment, for the same reason: `config`'s
resolved paths and `conventions`' parsed `kb/CONVENTIONS.md`. A test that *rewrites* the
conventions file mid-test calls `conventions.reset_cache()` itself - the fixture answers for the
boundary between tests, not for one inside a test.
## When to run ## When to run
Whenever you add or change a test under `tools/chemenu/tests/`. Whenever you add or change a test under `tools/chemenu/tests/`.
+149
View File
@@ -0,0 +1,149 @@
---
type: types/instruction.md
name: version-parts
description: Which part of the stack version a change bumps - the compatibility question (is the new version a drop-in replacement?) separated from the migration question (must existing content change?), plus what to do before a breaking bump.
---
# Pick the version part for a stack change
Two questions decide a version bump, and they are **not the same question**:
1. **Is the new version a drop-in replacement for the old one?** This is what the version
number itself says. Compatibility is read off the **leftmost non-zero component** - on this
stack (`2.x`) that is MAJOR, on a `0.x` stack it is MINOR. A bump that changes it is called
*boundary-crossing* below, because that is the term `version bump` and `docs verify` use in
their own messages.
2. **Must existing content be migrated?** This is a *consequence* a boundary crossing may or
may not have. `kb/` staying untouched does not make a change compatible, and
`version bump --no-migration` exists precisely because boundary-crossing bumps with an
untouched corpus are a real case.
Getting these backwards is how a genuinely breaking change ships as a MINOR. It happened once
already (see the case study at the end), which is why this file exists.
## When to run
Before every `tools/wikitool version bump` - the `stack-dev` skill's step 3 sends you here.
Read it in full the first time a change looks like it might be boundary-crossing; afterwards
the three-line test below is usually enough.
## Steps
1. **Apply the drop-in test.** The bump is boundary-crossing if **either** half fails:
- **Forward:** an existing instance can install the new machinery by copying `tools/`,
`types/`, `instructions/`, `AGENTS.md`, `VERSION` and `.wikitool-release.json` over itself,
and everything that worked before still works - with **no** hand-work by the user and **no**
migration script. Any step beyond the copy, however small, fails this half.
- **Backward:** having installed the new version, the user can put the old one back and be
where they started. A state file the old version cannot read, a rewritten corpus, a
renamed stamp - anything that makes the downgrade fail or leave a broken instance - fails
this half.
Content migration is one way to fail the forward half, not the definition of it.
2. **Check the catalogue** when the answer still feels like a judgment call. Each of these
crosses the boundary with `kb/` entirely untouched:
| What changed | Why the swap is not drop-in |
|---|---|
| The update path - `update_url`, the release feed, the repo it points at | The instance cannot repair its own `.wikitool-release.json`: it is machine-written, and invariant 1 forbids the hand-edit. The channel that would have told it to update is the channel that broke |
| The release artefact's name | Every download script and every pin against it breaks |
| The Python package's import name | `from <old> import ...` outside the shipped tree breaks |
| A command, subcommand, or flag that was removed or renamed | Scripts, CI workflows and instruction files calling the old spelling break |
| An environment variable's name | An instance configured through it silently loses the configuration |
| The shape of a machine-read file - `.wikitool-kb.json`, `.wikitool-release.json`, a generated index | The old version cannot read what the new one wrote, so the downgrade half fails even if the upgrade half passed |
| A type-spec's required fields | Existing pages stop validating - this one crosses *and* needs a content migration |
The catalogue is illustrative, not exhaustive. When something is not on it, go back to step 1.
3. **Otherwise pick the compatible part:**
| Change | Part |
|---|---|
| Fix, no interface change | `--patch` |
| New capability, drop-in in both directions | `--minor` |
4. **Stop and talk to the user before a boundary-crossing bump.** It is expensive in a way the
other two parts are not: every existing instance pays for it, once, by hand. Put in front of
them, in this order:
- **What breaks**, concretely - which file, which name, which call site.
- **What each existing instance must do**, as the steps they would actually run.
- **The alternatives**, so the break is a choice and not a side effect:
- *Avoid it* - keep the old name as an alias, read both file shapes, accept both flag
spellings. A compatibility shim carried for one release is usually cheaper than a
migration everyone runs.
- *Defer and batch it* - hold the break until the next boundary crossing, so instances pay
once instead of twice.
- *Split it* - ship the compatible half now, the breaking half later behind a deprecation
window that the changelog announces in advance.
- **Your recommendation**, with the trade-off named.
Then wait for an explicit go-ahead. Do not bump across the boundary on your own initiative.
5. **Record the break in the bump itself.** A boundary-crossing bump requires
`--breaking "<what breaks>"`, which writes a `**Breaking Change:**` line into the entry:
```bash
tools/wikitool version bump --major \
--title "<what changed>" \
--breaking "<what stops working, and what an instance must do about it>" \
--no-migration "<why no page has to change>" # only if that is true
```
`--breaking` is refused on a bump that crosses nothing, and required on one that does;
`docs verify` checks the newest boundary-crossing entry still carries the line. Write it for
the operator of an instance that has not read this repository: what stops working, and what
they do about it.
6. **Then answer the migration question separately.** Boundary-crossing and
content-migrating are independent:
- Content must change → write the migration document under `instructions/migrations/` per
[migrate-corpus.md](../migrate-corpus.md). `bump` finds it by its `migrates_to:` field.
- Content need not change → `--no-migration "<reason>"`, which records that in the entry.
Both are also needed by `docs verify`, for the same reason: an instance that learns it must
migrate, with nothing telling it how, is a dead end.
7. **Write the entry's body.** `bump` leaves it empty on purpose. A boundary-crossing entry
earns a paragraph that says *why this is breaking* - it is the one thing a future reader
cannot reconstruct from the diff, and it is what the next session in this position will read
instead of guessing.
## Decision points
- **The change ships no code - only `README.md`, `INSTALL.md`, `EVALS.md`, or `.gitea/`?** No
bump at all; CI's version gate is scoped to what changes behaviour.
- **A break you can see coming but are not making yet?** File it as an issue and let it
accumulate. Boundary crossings are cheaper in batches, and step 4's "defer" alternative is
only real if someone wrote the break down.
- **Unsure between MINOR and boundary-crossing?** It is boundary-crossing. The cost of an
unnecessary MAJOR is one extra release note; the cost of a MINOR that actually breaks is an
instance whose update path fails while its version number promised it would not.
- **The break only affects this repository, not a distributed instance** - something under
`instructions/dev/`, say? Then it is not a stack break at all: `dist export` never shipped it.
Judge by what an *exported* instance sees.
## Scope
Applies to the stack version in `VERSION` - `tools/`, `types/`, `instructions/`, `AGENTS.md`
and the contracts. It says nothing about the content shape in `.wikitool-kb.json`, which is
advanced by `wikitool migrate done` and described by [migrate-corpus.md](../migrate-corpus.md),
and nothing about wiki content operations, which are logged in `kb/log.md` and carry no version
at all.
Choosing the part remains a judgment call, deliberately: `docs verify` checks that a
boundary-crossing entry *documents* its break and its migration, never that the part was chosen
correctly. No validator can tell a renamed flag from a new one.
## Case study: 2.0.0
The Chemenu rebranding renamed the repo, the release artefact and the Python package. No page
in `kb/` changed, so the first attempt was `1.9.0` - the migration question, answered correctly,
substituted for the compatibility question, which was never asked. Three things broke: every
existing instance's `update_url` pointed at a repo path that no longer existed and could not be
hand-repaired; the artefact name changed; the import name changed. The correct bump was
`--major --no-migration`, and the `CHANGES.md` entry for `2.0.0` carries the reasoning in full
under "Warum das trotzdem MAJOR ist". The error was caught by the user, not by the
documentation - which is what step 4 is for.
+45 -4
View File
@@ -22,10 +22,11 @@ Read the exit code first - it says which of these applies:
## Exit 42: user clearance required ## Exit 42: user clearance required
A `wikitool` command that exits **42** is not reporting an error. It is refusing to act until a A `wikitool` command that exits **42** is not reporting an error. It is refusing to act until a
human has *read its output*. Two gates use it today - the Mass-Update Gate (`publish`, on a human has *read its output*. Three gates use it today - the Mass-Update Gate (`publish`, on a
change touching 10 or more counted files) and the rebase-review gate (`sync` and `publish`, on change touching 10 or more counted files), the rebase-review gate (`sync` and `publish`, on
a rebase whose incoming commits touch a file this session is also changing) - but the rule is a rebase whose incoming commits touch a file this session is also changing), and the
about the exit code, not the command: Publish-Remote Gate (`publish`, on a push to a target this checkout has not declared) - but the
rule is about the exit code, not the command:
> **Copy the command's output into your reply - the substance of it, not a description of it - > **Copy the command's output into your reply - the substance of it, not a description of it -
> and stop.** Run no further commands in that turn. > and stop.** Run no further commands in that turn.
@@ -62,6 +63,46 @@ clearance.
Background: [[Mass-Update Gate]] (`kb/concepts/Mass-Update Gate.md`). Background: [[Mass-Update Gate]] (`kb/concepts/Mass-Update Gate.md`).
### Publish-Remote Gate
The Mass-Update Gate asks whether a change is too large to publish. This one asks the question
underneath it: **whether this is the right repository to publish to at all.**
A checkout that holds private content usually has two remotes - its own, and the public upstream
it takes stack updates from. Git does not distinguish them at push time, so one wrong `--remote`
puts a private corpus on a public repository. That is not cheaply reversible: a force-push moves
the branch, but the objects stay fetchable by SHA until someone expires the server's reflogs and
runs `git gc --prune=now` on the bare repo.
`.wikitool-remotes.json` names the push URLs a checkout permits:
```json
{ "schema": 1, "allowed_push_urls": ["ssh://git@example.net:22/you/your-wiki.git"] }
```
It pins **URLs, not remote names** - a name-based list would wave through a `publish` whose
`origin` had been repointed, which is the failure it exists to catch. It reads the remote's
`pushurl` when one is set, because that is where `git push` actually writes.
The file is per-checkout and gitignored, for the same reason `ENVIRONMENT.md` is: two clones push
to two different places, so a committed copy would tell a private clone that the public upstream
is a legitimate target for its own content. **Absent means unrestricted** - a single-remote
checkout with nothing private in it has nothing to protect, and `doctor` reports which state a
checkout is in, WARNing only when there is more than one remote and no allowlist. A malformed
file is an error rather than "no restriction": a corrupted safeguard must not read as a disabled
one.
**This gate has no `--confirm` token, on purpose.** The other two clear with a token because the
question they ask ("is this change right?") is one the agent can put to the user and the user can
answer for that one changeset. This one asks "does this content belong to that repository?", which
is a standing property of the checkout, not a per-push judgment. The way past it is for the user
to add the URL to the file. **An agent must never edit `.wikitool-remotes.json` to get past a
refusal** - that is opening a gate on your own initiative, which AGENTS.md invariant 6 forbids.
The setup this gate exists for - a private instance that takes stack updates from a public
upstream - is [private-instance.md](private-instance.md). Step 4 there arms it, deliberately
*before* the first `publish`: added afterwards it leaves open exactly the window it closes.
## Iteration Budget Gate and loop-breaker ## Iteration Budget Gate and loop-breaker
Every `wikitool` call is counted per session. Calls are refused past **60 in a session**, or Every `wikitool` call is counted per session. Calls are refused past **60 in a session**, or
+10 -6
View File
@@ -7,9 +7,13 @@ manual: true
# German terminology for `kb/` # German terminology for `kb/`
Reference vocabulary for [kb/CONTRACT.md](../kb/CONTRACT.md#language)'s rule that pages are Reference vocabulary for [kb/CONVENTIONS.md](../kb/CONVENTIONS.md#language)'s rule that this
written in German. The rule lives there; the word list lives here, because it is lookup material instance's pages are written in German. The rule lives there; the word list lives here, because
rather than a norm and would otherwise be loaded on every write. it is lookup material rather than a norm and would otherwise be loaded on every write.
**This file belongs to the `german` language profile, not to the stack.** An instance writing in
another language deletes or replaces it - see
[kb-profiles.md](kb-profiles.md).
Derived from translating all 248 pages on 2026-08-29. Every entry below is a decision that was Derived from translating all 248 pages on 2026-08-29. Every entry below is a decision that was
made wrong at least once first - each cost a correction pass across published pages, which is why made wrong at least once first - each cost a correction pass across published pages, which is why
@@ -98,8 +102,8 @@ none of them structural, so no check found them. It is the one thing to watch fo
instructional prose. instructional prose.
- **Quotations are never reworded**, neither translated nor moved into the impersonal register. - **Quotations are never reworded**, neither translated nor moved into the impersonal register.
- Buzzwords and AI filler are banned by [kb/CONTRACT.md](../kb/CONTRACT.md#tone); the German list - Buzzwords and AI filler are banned by [kb/CONVENTIONS.md](../kb/CONVENTIONS.md#tone); the
is there. German list is there.
- Dash as ` - `, not `—`. - Dash as ` - `, not `—`.
- German number formatting only in prose („10.000 Punkte"). Never inside code, version numbers or - German number formatting only in prose („10.000 Punkte"). Never inside code, version numbers or
measurements (`75-85 px`, `10m`, `0.90`). measurements (`75-85 px`, `10m`, `0.90`).
@@ -108,4 +112,4 @@ instructional prose.
This is about prose in `kb/`. What is prose and what is an identifier - titles, headings, wikilink This is about prose in `kb/`. What is prose and what is an identifier - titles, headings, wikilink
targets, cite-ids, enum values, tags, code - is decided by targets, cite-ids, enum values, tags, code - is decided by
[kb/CONTRACT.md](../kb/CONTRACT.md#language), not here. [kb/CONTRACT.md](../kb/CONTRACT.md#language-and-identifiers), not here.
+191
View File
@@ -0,0 +1,191 @@
---
type: types/instruction.md
name: kb-profiles
description: Ready-made answers for kb/CONVENTIONS.md and each COLLECTION.md - the proven collection contracts and language profiles this stack has shipped, offered as a palette to adopt or adapt, never as a binding source.
manual: true
---
# Pick a profile for a collection or for this instance's conventions
**This page is a palette, not an enum.** Each `kb/<name>/COLLECTION.md` stays authoritative for
its own collection and `kb/CONVENTIONS.md` for the instance as a whole; an entry here is a
proven starting point, nothing more. Adopting one means *copying its text into* that file - not
pointing at this page and inheriting whatever it says later. Nothing in the stack reads this
document, and `profile:` in a contract's frontmatter records where the text came from, not where
it lives.
That direction is deliberate and it is the opposite of how this repo used to work. Language,
tone, naming and the relationship vocabulary sat in `kb/CONTRACT.md`, a file `dist export` ships
verbatim - so every instance that wanted something else edited a stack file, and an upstream
merge handed the stack's answer back. What binds is now the instance's; what ships is this
catalogue, and it binds nothing.
## When to run
- Setting up a new instance: the KB-language step of
[setup-instance.md](setup-instance.md) sends you here to fill `kb/CONVENTIONS.md`.
- Adding a collection to an existing instance, and wanting a contract that already works rather
than a blank one.
- Rewriting an existing `COLLECTION.md` or `kb/CONVENTIONS.md` and wanting to see what the
alternatives were.
Not for changing what the *stack* enforces. That is [kb/CONTRACT.md](../kb/CONTRACT.md), and it
is not a profile.
## Steps
1. **Decide what you are filling.** Two different files, and they are not interchangeable:
| File | Holds | Profiles below |
|---|---|---|
| `kb/CONVENTIONS.md` | Language, section headings, naming forms, tone, relationship labels, confidence rubric - once per instance | [Language profiles](#language-profiles) |
| `kb/<name>/COLLECTION.md` | What one collection holds, its quality goal, its local linking and naming rules | [Collection profiles](#collection-profiles) |
2. **Copy the entry's text into the file**, then edit it until it is true of this instance.
A profile you adopted and then changed is still that profile's `profile:` value - the field
records the starting point, not a promise of fidelity.
3. **Record it.** `profile: <name>` in the file's frontmatter, or `profile: none` for a
collection written from scratch. `wikitool docs verify` checks the field is there; it does
not check the value against this page, because a collection an instance invented has no
entry here to name.
4. **Set `required_by_stack:` on a collection - and set it correctly.** This one is *not* a
choice: it says whether `wikitool` resolves against the collection by name, and
`docs verify` checks it against the stack's own list. `sources` is `true`, everything else
is `false`. See [kb/CONTRACT.md § Collections](../kb/CONTRACT.md#collections).
## Language profiles
A language profile answers all of `kb/CONVENTIONS.md` at once. There is one today, because one
is what this repo has actually run.
### `german`
The profile this repo's own instance uses, and the reason this catalogue exists: it was the
stack's hardcoded behaviour until the conventions file existed.
| Decides | Value |
|---|---|
| `language:` | `de` |
| `sections:` | `Beziehungen` / `Siehe auch` / `Fußnoten` |
| Naming | Human-readable titles with spaces; singular for entities; `adr-NNN-` for decisions; `X vs Y` for comparisons |
| Tone | Wikipedia register, with a German buzzword and filler list |
| Relationship labels | `hängt ab von` · `verwendet` · `implementiert` · `erweitert` · `ersetzt` · `steht in Konflikt mit` · `benötigt` · `erzeugt` · `konsumiert` · `besitzt` · `pflegt` · `läuft auf` · `verwandt mit` |
| Confidence rubric | 0.5 base, +0.2 per supporting source (max +0.6), recency and source-quality bonuses; hedge with "möglicherweise"/"kann" below 0.6, "unsicher"/"unbestätigt" below 0.4 |
| Terminology | [german-terminology.md](german-terminology.md) - which English terms stay English, and which have a settled German form |
**The full text to copy** is this repo's own [kb/CONVENTIONS.md](../kb/CONVENTIONS.md). An
instance adopting it takes that file, not this table; the table is what the profile *decides*,
so you can tell at a glance whether it is the one you want.
Adopting it also means keeping `german-terminology.md`. An instance on any other language
deletes or replaces that file - it is the profile's lookup material, not the stack's.
### `english`
What `kb/CONVENTIONS.md.template` ships as its default, so "adopt `english`" means "fill in the
template and change nothing structural". `sections:` are `Relationships` / `See Also` /
`Footnotes`, which are also the names this stack wrote before it had a conventions file - so a
corpus that predates the split needs no translation pass to adopt this profile.
There is no worked text for the rest of it. The template's placeholders are the questions;
`german` above is what a filled answer looks like.
### Writing a third one
A language profile is not a translation of `german`. Two of its sections are judgment about a
language rather than vocabulary in it - which foreign technical terms stay untranslated, and how
to hedge a low-confidence claim - and those are exactly the two that read as awkward when
translated mechanically. Write them, do not convert them.
The one part that is mechanical: `section_aliases:`. Whatever the corpus used before goes in
that list, and the pages then migrate one at a time instead of all at once.
## Collection profiles
The four collections this repo runs. Each is a whole `COLLECTION.md`, and **the text to copy is
the file itself** - `dist export` ships each one as `kb/<name>/COLLECTION.md.template`, which a
new instance adopts by renaming. What follows is what each decides, so you can tell whether you
want it.
### `entities`
Concrete, pointable things: projects, deployed systems, tools, technologies, people.
- **Quality goal:** pointability plus currency - what the thing is, where it actually is, and
whether that is still true.
- **Areas** driven by the `entity_type:` field: `projects/`, `systems/`, `tools/`,
`technologies/`, `people/`. Areas, not collections - they inherit the contract and carry no
`COLLECTION.md`.
- **Per-area emphasis** spelled out, so a system page is not written like a technology page.
- `required_by_stack: false`.
Take it when the wiki is about things that exist. Adapt the area list first: it is the part most
likely to be wrong for another domain.
### `concepts`
Ideas rather than things: architectures, patterns, protocols, workflows, recurring problems,
and the decisions taken about them.
- **Quality goal:** explanatory sufficiency - the page answers *why it is done this way* without
the reader opening the entity pages that use it.
- Carries the **ADR shape**: context, decision, consequences, status, and the rule that a
superseded decision is never rewritten.
- Routes head-to-head arguments out to `comparisons/` rather than hosting them.
- `required_by_stack: false`.
Take it whenever `entities` is taken - the split between the two is what keeps either from
becoming an essay.
### `sources`
One page per ingested source, carrying the `raw_files:` provenance every citation resolves
against.
- **Quality goal:** faithful compression - what *this source* said, not what was concluded from
it. A source page improved beyond its source is no longer evidence.
- Titles carry the `Source - ` prefix, applied by `wikitool new source`.
- `required_by_stack: **true**`. `sources coverage`, `[^cite-id]` resolution and
`kb/provenance.md` resolve against the name `sources`.
Not optional in the way the others are. An instance may rewrite its authoring rules and may not
rename or drop it.
### `comparisons`
Structured head-to-head evaluations of two or more things that already have pages.
- **Quality goal:** decidability - named, checkable dimensions and a stated trade-off, so a
reader with a concrete situation can choose.
- Every subject must already have a page; a comparison is a view over existing knowledge.
- **Exempt from the orphan check** - comparisons are reached through the catalog, not through
inbound prose links.
- `required_by_stack: false`.
Skip it in a wiki that records rather than decides. It is the one of the four that is genuinely
optional.
## Decision points
- **A profile is almost right?** Copy and edit. There is no partial adoption and no override
file - the copy *is* the mechanism, and `profile:` still records where it started.
- **Two collections want the same profile?** Fine. `profile:` is not unique, and two
collections holding different subject matter under the same authoring rules is an ordinary
outcome.
- **Changing `sections:` after pages exist?** That is a corpus migration, not an edit. Put the
old names in `section_aliases:` first, then translate page by page - the tool keeps finding
the old headings for as long as the alias stands. See
[migrate-corpus.md](migrate-corpus.md).
- **Tempted to make this page binding** - to have `COLLECTION.md` say `profile: entities` and
nothing else? Do not. That is the arrangement this split was written to end: the instance
would be bound by a file the stack ships and upgrades, which is how an upstream merge changes
an instance's authoring rules without anyone deciding to.
## Scope
Covers what an instance authors under `kb/`. It says nothing about what the stack enforces
([kb/CONTRACT.md](../kb/CONTRACT.md)), what a page structurally is
([types/type-spec.md](../types/type-spec.md)), or how a command behaves
([tools/CONTRACT.md](../tools/CONTRACT.md)). None of those are profiles, and none of them are
the instance's to change.
+199
View File
@@ -0,0 +1,199 @@
---
type: types/instruction.md
name: link-taxonomy
description: The link-label catalogue - every relationship label a page may declare in related:, grouped by register, with the reader need each one names. A palette to authorise from in a COLLECTION.md, never binding on its own.
manual: true
---
# Pick a link label
**This page is a palette, not an enum.** It lists every label this stack ships with and what
each one asserts. What a page may actually *use* is decided by its own collection: each
`kb/<name>/COLLECTION.md` authorises a subset per destination, and `wikitool lint` checks
`related:` against that authorisation rather than against this file. A collection that
authorises six labels has six, however long this list gets.
A label is an **identifier, not prose**. It is written into `related:` as a machine value and
rendered verbatim into the page body, so it is never translated - not in a German wiki, not in
any other. Which words a page is *written* in stays [kb/CONVENTIONS.md](../kb/CONVENTIONS.md)'s;
this is not one of them.
## The invariant every label obeys
Every label completes, with the page carrying the link as the grammatical subject:
> `[source] <label> [target]`
The page containing the link asserts something **about** the target. `Hermes depends-on
PostgreSQL` reads correctly on Hermes' page; the same fact written on PostgreSQL's page is a
different label (`required-by`), not the same one pointing back. Omitted helper verbs ("is",
"a") are fine where they do not reverse the endpoints.
This is Commonplace's ADR-058, adopted wholesale, and it is what makes a label checkable rather
than a matter of taste: read the sentence out loud, and if it says the opposite of what you
meant, the label is wrong.
## Direction is authored, never mirrored
Each direction is a separate decision. A link back from the target is welcome when it
independently helps a reader *there* - and unnecessary when it does not. **Do not add a reverse
edge merely to mirror the first one.** The inbound view is rendered from the graph by
`index rebuild` and `search`, so a reader landing on the target sees what points at it whether
or not anyone wrote a second edge.
That is why most labels below have no inverse. Only two pairs do, because in each the reverse
direction is a genuine primary statement someone would write on its own: `depends-on` /
`required-by` and `runs-on` / `hosts`.
## When to run
Adding or changing a `related:` entry, authorising labels in a `COLLECTION.md`, or judging
whether a relationship is worth naming as a formal edge at all.
## Steps
1. **Decide whether this is an edge.** Not every mention is one. An edge is a reader aid: it
says *follow this if you need X*. A subject mentioned once in passing is prose with a
`[[wikilink]]`, not a declared relationship. Over-declaring is how a graph becomes a list of
everything adjacent to everything.
2. **Say the sentence.** `[this page] <label> [that page]`. If it reads backwards, you want the
other page to carry the edge, or a different label.
3. **Pick from the register that fits the pair**, below. Prefer the most specific label that is
true; fall back outward only when nothing fits.
4. **Check the collection authorises it** for that destination -
`kb/<name>/COLLECTION.md`'s `outbound:` block. If the label you want is not authorised and
should be, that is a collection-contract change, made deliberately, not a lint error to
route around.
5. **Write it with the tool**, never by hand:
```bash
tools/wikitool xref add --a "<This Page>" --b "<That Page>" --rel <label>
```
## The catalogue
### Operational
Concrete things and how they stand to one another - the register this instance runs on. Mostly
entity to entity.
| label | inverse | asserts |
|---|---|---|
| `depends-on` | `required-by` | cannot function without the target |
| `required-by` | `depends-on` | the target cannot function without this |
| `runs-on` | `hosts` | executes on the target as its substrate |
| `hosts` | `runs-on` | provides the substrate the target executes on |
| `uses` | — | employs the target at runtime, but survives without it |
| `produces` | — | emits the target as an artifact or data |
| `consumes` | — | reads the target as an artifact or data |
| `maintains` | — | carries the upkeep of the target |
| `owns` | — | is accountable for the target's existence and decisions |
`uses` versus `depends-on` is the distinction worth keeping sharp: if removing the target breaks
this thing, it is `depends-on`. `owns` versus `maintains`: accountability versus labour, and
they are often different people.
### Realization
How an idea becomes a running thing. Usually concept to entity or the reverse.
| label | asserts |
|---|---|
| `implements` | is a concrete realization of the target |
| `operationalized-from` | is the prescriptive form of the target's theory |
| `mechanism` | is the mechanism by which the target works |
| `procedure` | is the procedure for carrying out the target |
| `applies-when` | applies under the condition the target describes |
| `operates-on` | acts upon the target as its subject matter |
| `invokes` | calls the target as a step within itself |
### Conceptual
Inference and comparison between ideas.
| label | asserts |
|---|---|
| `extends` | develops the target's argument further |
| `grounds` | provides the basis the target rests on |
| `rests-on` | takes the target as its premise |
| `enables` | is the operational prerequisite that makes the target possible |
| `precondition` | must hold before the target applies |
| `exemplifies` | is an instance of the general claim the target makes |
| `abstracted-from` | generalizes from the target |
| `contrasts` | differs from the target in a way worth reading both for |
| `compares-with` | is weighed against the target on shared dimensions |
| `contradicts` | asserts something the target denies |
| `composition` | is composed of the target |
| `part-of` | is a component of the target |
`grounds` / `rests-on` is a genuine pair and both directions are primary statements; they are
listed separately rather than as inverses because either page may legitimately carry only its
own side.
### Lineage
Where something came from, and what replaced it.
| label | asserts |
|---|---|
| `supersedes` | replaces the target, which is now historical |
| `derived-from` | was produced from the target |
| `adapted-from` | was reworked from the target for a different purpose |
| `defined-in` | takes its definition from the target |
A superseded page is never deleted or rewritten - see the collection contract for
`kb/concepts/`.
### Evidence
The provenance register. Distinct from `sources:` and `[^cite-id]`, which are the *mechanical*
provenance path: these two are authored claims about how strongly something is backed.
| label | asserts |
|---|---|
| `evidenced-by` | is supported by the target as evidence |
| `is-evidence-for` | serves as evidence for the target's claim |
### Universal
| label | asserts |
|---|---|
| `see-also` | nothing more specific applies, and a reader here would still want the target |
**`see-also` is the last resort and should stay rare.** A collection where it is the commonest
label has a vocabulary problem, not a lot of loosely related pages. The previous vocabulary's
`verwandt mit` was exactly that, and it is the reason this catalogue exists.
## Extending it
Adding a label is a line of data, never a code change:
1. Add a row here, in the register it belongs to, with the sentence it completes.
2. Authorise it in the `COLLECTION.md` of every collection that may use it.
The registers are advisory groupings for readers, not a schema - nothing checks that a label is
used only within its register. Invent an intra-collection label the work needs and propose it
here afterwards; the architecture is deliberately loose, because the link theory is still
developing.
## Decision points
- **Two labels both fit?** Take the more specific one. If they are equally specific and mean
different things, the relationship is probably two edges.
- **The relationship reads better from the other page?** Write it there. Nothing is lost - the
inbound view renders it here.
- **You want a reverse edge for navigation?** You do not need one. That is what the rendered
inbound view is for, and it is complete in a way an authored mirror never was.
- **Nothing fits at all?** Use `see-also` and say so in the commit, or propose a label. Do not
stretch a label whose sentence reads false - a wrong edge is worse than a weak one, because
it is machine-readable and will be believed.
## Scope
Covers labels on `related:` edges between pages. Says nothing about `sources:` (the provenance
field, unlabelled by construction), `[^cite-id]` footnotes
([kb/CONTRACT.md](../kb/CONTRACT.md#provenance-and-citation)), or `tags:` (search keys, not
relationships).
+108
View File
@@ -0,0 +1,108 @@
---
type: types/instruction.md
name: mcp-read-server
description: Run and keep current the MCP read server that serves this wiki to a second consumer
---
# Running the MCP read server
Chemenu has a second consumer. `search`, `types`, `describe_type`, `lint` and `status` are
served over MCP to callers that are not this terminal - the CLI and the server are two adapters
over one core (`chemenu.api.Corpus`), not a CLI with a network interface bolted on.
This document is about *operating* it: how to start it, what has to be true of the checkout it
serves, and how that checkout stays current. What it exposes and why is in
[tools/CONTRACT.md](../tools/CONTRACT.md) and in the module's own docstring
(`tools/chemenu/mcp/server.py`).
**Deployment is deliberately not here.** Which cluster, which ingress host, where the credential
lives - that is private infrastructure and this is a public repository. What is here is
everything an operator needs that is *true of the software* rather than of one installation.
## When to run
- Standing something up for a consumer that is not a terminal on this machine.
- Diagnosing an answer that looks stale, or one that disagrees with the CLI.
- Before pointing a new consumer at an existing server.
## Steps
1. **Install the server's dependency.** It is deliberately not in `requirements.txt`: an
instance that only uses the CLI should not be made to install pydantic, starlette, uvicorn
and cryptography to do it.
```bash
tools/.venv/bin/pip install -r tools/requirements-mcp.txt
```
2. **Decide which checkout it serves.** The root resolves by precedence - an explicit `--root`,
then `$CHEMENU_ROOT`, then the checkout the package lives in. A deployment points at its
corpus with one variable and no code:
```bash
export CHEMENU_ROOT=/srv/chemenu
```
3. **Take tracing out of the served tree.** The server refuses to start otherwise, and the
refusal is the point: telemetry defaults to on and writes under `reports/telemetry/` inside
the repo, which step 5's sync is entitled to wipe. Either is fine:
```bash
export WIKI_TRACE=0 # off
export WIKI_TRACE_DIR=/var/log/chemenu # or elsewhere, outside the corpus
```
4. **Start it on the transport that matches what is in front of it.**
```bash
tools/.venv/bin/python -m chemenu.mcp # stdio
tools/.venv/bin/python -m chemenu.mcp --transport streamable-http # deployed
```
`stdio` is for developing and testing without a network - one process per consumer, started
locally. `streamable-http` is what a deployed instance speaks, and the only one the
authentication middleware can sit in front of, because that middleware is an HTTP reverse
proxy. `sse` is reachable through the SDK and deliberately not offered: it is the superseded
remote transport, and building on it now only moves the migration later.
5. **Keep the checkout current by polling, and keep it clean.**
```bash
git -C "$CHEMENU_ROOT" fetch --quiet origin && \
git -C "$CHEMENU_ROOT" reset --hard --quiet origin/main
```
Every few minutes, from a timer beside the server. Polling rather than a webhook on purpose:
it needs no inbound endpoint and no signature checking, which is a smaller surface than the
thing it would optimize. A webhook is a later optimization, not a starting point.
`reset --hard` is load-bearing, not a convenience. The corpus cache reuses a parse while the
commit is unchanged and **refuses to cache a dirty tree at all**, so a checkout that has
drifted answers correctly but reparses on every request - and every answer it gives is
stamped `"commit": null`, because a dirty tree corresponds to no revision.
## Decision points
- **An answer looks stale?** Read `commit` in the response. If it names an old revision, the
sync is not running. If it is `null`, the served tree has uncommitted changes - something is
writing into the corpus that should not be.
- **The server disagrees with `wikitool` on the same query?** That is a defect, not a
configuration difference: the two go through the same functions and a golden test holds their
output together (`tools/chemenu/tests/test_mcp_server.py`). Check first that both are pointed
at the same root - `CHEMENU_ROOT` is easy to set for one and not the other.
- **Asked to expose a write tool?** There is none, and the way to add one is not a flag. The
server imports nothing under `chemenu.commands`, so `new`, `touch`, `xref`, `cite`, `publish`
and `migrate` are unreachable from it rather than filtered out of a list. Submitting documents
from outside is a different design with a quarantine in it - Gitea #32 - not a tool added
here.
- **Asked to rate-limit inside the server?** Rate limiting belongs in the middleware in front of
the process, next to authentication. Not the Iteration Budget Gate: that exists to stop an
agent *session* from iterating unnoticed over the wiki's state, which is why retrieval is
exempt from it, and using it as a rate limiter would dilute it into one.
## Scope
Not for setting up an instance ([setup-instance.md](setup-instance.md)) or a fresh clone
([bootstrap.md](bootstrap.md)). Not for the authentication or rate-limiting middleware, which is
infrastructure configuration rather than part of this repository. Not a write path: see the
decision point above.
@@ -0,0 +1,150 @@
---
type: types/instruction.md
name: 3.0.0-authoring-conventions
description: 'Adopt the instance-owned authoring conventions introduced in 3.0.0 - write kb/CONVENTIONS.md, declare profile:/required_by_stack: on every COLLECTION.md, and replace kb/CONTRACT.md with the shipped one.'
manual: true
migrates_to: 3.0.0
migration_kind: mechanical
---
# Adopt this instance's own authoring conventions (3.0.0)
Until 3.0.0, the rules for writing a page were split by *location*: everything about `kb/` sat
in `kb/CONTRACT.md`, a file every distribution ships verbatim. Half of it was never the stack's
to decide - the language pages are written in, the three tool-owned section headings, the naming
forms, the tone, the relationship labels, the confidence rubric - so an instance that wanted
something else edited a file the stack also ships, and an upstream merge handed the stack's
answer back.
3.0.0 splits it by *ownership* instead. `kb/CONTRACT.md` keeps only what `wikitool` enforces;
everything else moves into a new `kb/CONVENTIONS.md` that belongs to this instance, and each
`kb/<name>/COLLECTION.md` now declares what it is. The compiler reads its section headings from
that file rather than from `tools/chemenu/sections.py`.
**No page changes.** Not one line under `kb/entities/`, `kb/concepts/`, `kb/sources/` or
`kb/comparisons/` is touched. What changes are the contracts beside them, which is why this is
`mechanical` and takes minutes rather than a workshop.
## When to run
After installing 3.0.0 machinery over an instance that was on 2.x, when `tools/wikitool doctor`
reports `FAIL conventions` or `tools/wikitool docs verify` reports a `COLLECTION.md` with no
frontmatter. `tools/wikitool migrate status` names this document.
**Until it has run, the compiler answers out of a fallback.** `xref add` and `cite add` write
`## Beziehungen` / `## Siehe auch` / `## Fußnoten` - what this stack hardcoded before the
conventions file existed. That is correct for a corpus written under them and wrong for any
other, so run this before the next `wiki-ingest` or `wiki-manage`, not afterwards.
## Steps
1. **Replace `kb/CONTRACT.md` from the release.** It is machinery that happens to live under a
content directory, and the tarball update path used to skip it (see `INSTALL.md`, which now
names it explicitly). The 3.0.0 version is roughly half the length of the 2.x one - the
removed half is what step 2 is about to write into a file of yours.
```bash
cp <unpacked-release>/kb/CONTRACT.md kb/CONTRACT.md
```
A private instance cloned from an upstream takes it with the merge instead - see
[private-instance.md](../private-instance.md), whose update procedure now re-takes the
upstream side for exactly this path.
2. **Write `kb/CONVENTIONS.md`.** Two ways in, and the first is almost always right:
- **This instance writes German pages** (it did, unless you changed it): copy the release's
`kb/CONVENTIONS.md.template` and fill it from the `german` profile in
[kb-profiles.md](../kb-profiles.md) - whose worked full text is the origin repo's own
`kb/CONVENTIONS.md`. Everything in it was already true of your corpus; it was simply
written down somewhere you did not own.
- **You had changed the language**, and therefore hold local edits to `kb/CONTRACT.md`,
`types/*.md` and `tools/chemenu/sections.py`: those edits are what this file replaces. Copy
the canonical heading names out of your old `sections.py` into `sections:`, the labels and
tone rules out of your old `kb/CONTRACT.md`, then **discard the local edits under `tools/`
and `types/`** and take the shipped versions. That is the whole point of the change: there
is nothing left to patch there.
The minimum the tool needs is the frontmatter:
```yaml
---
language: de
profile: german
sections:
relationships: Beziehungen
see_also: Siehe auch
footnotes: Fußnoten
---
```
Set `sections:` to the names **your existing pages already carry**, not to what you would
prefer. Changing them is a separate, real corpus migration; `section_aliases:` is how it is
done page by page ([migrate-corpus.md](../migrate-corpus.md)).
Drop the `wikitool:template-unfilled` sentinel line while filling it in - `doctor` FAILs on a
renamed-but-unanswered template exactly as it does for `USER.md`.
3. **Declare each collection.** Two frontmatter lines at the top of every
`kb/<name>/COLLECTION.md`:
```yaml
---
profile: <the entry in instructions/kb-profiles.md this contract came from, or none>
required_by_stack: false
---
```
`required_by_stack: true` on `kb/sources/` and **nowhere else**. It is not a preference:
`sources coverage`, `[^cite-id]` resolution and `kb/provenance.md` resolve against that name,
and `docs verify` checks the field against the stack's own list in both directions.
For the four default collections, the shipped `kb/<name>/COLLECTION.md.template` files carry
the right values already.
4. **Verify.** All three must pass:
```bash
tools/wikitool doctor # `conventions` must be OK
tools/wikitool docs verify
tools/wikitool lint
```
`migrate verify` is deliberately not in that list: it compares pages, and no page changed.
Running it would report nothing and prove nothing.
5. **Record it.**
```bash
tools/wikitool migrate done 3.0.0 --pages 0
```
`--pages 0` is honest, not a placeholder - see the note under step 1.
## How to tell a migrated instance from an unmigrated one
`kb/CONVENTIONS.md` exists, carries no `wikitool:template-unfilled` line, and names all three
slots under `sections:`; every `kb/*/COLLECTION.md` opens with a frontmatter block; and
`kb/CONTRACT.md` has a `## Language and identifiers` heading rather than a `## Language` one.
`doctor` answers all of that in one call.
## Decision points
- **`doctor` says `conventions: FAIL` after step 2?** It prints which slot is missing. The three
keys are `relationships`, `see_also` and `footnotes` - the *slot* names are fixed, only their
values are yours.
- **A collection this instance invented, with no profile behind it?** `profile: none`. The field
records where the text came from; it is free text and `docs verify` does not check it against
the catalogue, because an invented collection has no entry there to name.
- **Tempted to point `profile:` at the catalogue instead of copying the text?** Do not. An
adopted profile is a copy; a reference would put your binding authoring rules in a file the
stack ships and upgrades, which is the arrangement 3.0.0 exists to end.
- **Your old `kb/CONTRACT.md` had local edits you still want?** They belong in
`kb/CONVENTIONS.md` now. If something you edited has no home there, it was a stack rule you
overrode - file it as an issue against the origin repo rather than re-applying it.
## Scope
One instance's contracts, once. It changes no page, no frontmatter on a page, and nothing under
`raw/`. The machinery half of the 3.0.0 upgrade - copying `tools/`, `types/`, `instructions/`,
`AGENTS.md`, `VERSION` and `.wikitool-release.json` - is `INSTALL.md`'s, and has to have
happened before step 1.
@@ -0,0 +1,146 @@
---
type: types/instruction.md
name: 4.0.0-link-taxonomy
description: Move every relationship from free-text prose in a body bullet to a labelled edge in related:, and every tool-owned body region from heading-matching to a marker pair.
manual: true
migrates_to: 4.0.0
migration_kind: assisted
obligation: required
---
# Move relationships into the data, and generated regions behind markers (4.0.0)
Until 4.0.0 the stack used **prose as an identifier** in three places, and each one cost
something measurable:
| Was the identifier | Cost |
|---|---|
| A section's heading text (`## Beziehungen`) | The KB language was a compiler constant, and the region's *end* was a guess. Content sitting after it was silently deleted on eight pages |
| A relationship label in a body bullet (`- **hängt ab von:**`) | Nothing could check the vocabulary, so it drifted to **152 distinct labels** across 337 bullets against thirteen that were documented |
| The reciprocal half of every edge | `xref add` mirrored every link, which made per-collection label authorisation impossible and filled `## Siehe auch` with 555 unlabelled bullets, 353 of them provably redundant |
4.0.0 replaces all three. A region is delimited by a marker pair and rendered from frontmatter;
a label is a machine value in `related:`, drawn from a catalogue and authorised per destination
by the source collection; an edge is authored in one direction and the inbound view is computed.
**This one touches pages.** Unlike 3.0.0 it is not a contract reshuffle: every `related:` entry
and every tool-owned body region changes. It is `assisted` because there is no mapping table -
mapping free-text German onto a 35-label catalogue is a judgment call per edge, and a large
minority of the old labels are reverse directions that under the new model are not stored at all.
## When to run
After installing 4.0.0 over an instance on 3.x. `tools/wikitool migrate status` names it, and
`lint` reports `unlabelled_edges` for every unconverted edge - that count reaching zero is how
you know the run is finished.
**Nothing breaks while it is outstanding.** Unlabelled edges and undelimited regions are read,
not rejected: `links.py` treats a bare title as an edge whose label is not declared yet, and
`provenance.split_cite_block` falls back to the pre-marker layout. That is deliberate - a corpus
has to stay readable while it is being converted - and it is why the two lint findings are
advisory until step 6 promotes them.
## Steps
1. **Rewrite `kb/CONVENTIONS.md`'s `sections:` block.** Three slots become two, because the
See Also region is gone:
```yaml
sections:
links: <your heading for declared relationships>
footnotes: <your heading for citation definitions>
```
Delete `section_aliases:` if you have one - nothing matches on heading text any more, so
there is nothing to alias. The heading is now a *rendering* value: changing it re-renders
the words above each region on the next write and can no longer split a page.
2. **Add an `outbound:` block to every `kb/<name>/COLLECTION.md`.** Which labels a page may use,
per destination collection, with `any` as a wildcard:
```yaml
outbound:
entities: [depends-on, runs-on, uses, see-also]
concepts: [implements, see-also]
```
The catalogue to draw from is [link-taxonomy.md](../link-taxonomy.md); the four contracts in
the origin repo are worked examples. **The source collection decides** - that is what makes a
35-label palette usable, and it is why the reverse edge can no longer be written
automatically. A destination you do not list authorises nothing, which is a real answer.
3. **Fix your page type-spec templates.** If you adopted the 3.0.0 templates, they contain
`## {section.relationships}` and `## {section.see_also}`. Those variables no longer exist and
would be written into new pages literally. **Delete both sections from the `## Template`
block** - a template must not scaffold a tool-owned region at all: it is generated between
markers on the first `xref add` / `cite add` and re-rendered on every write.
4. **Convert the corpus**, following [migrate-corpus.md](../migrate-corpus.md). Cut it into
units sized against the iteration budget; the origin repo used four, ~45 pages each. Per page:
- For each labelled bullet under the old relationships heading: say the sentence
`[this page] <label> [target]` and pick the catalogue label that makes it true. If it only
reads true **backwards**, the edge belongs on the other page - move it there rather than
inventing an inverse label the catalogue does not have.
- For each bare `- [[X]]` bullet under the old See Also heading: drop it if a labelled edge
already connects the pair. Otherwise decide - a real label, or dropped with the reason
recorded. **Do not convert them to `see-also` in bulk.** That is the one shortcut this
migration explicitly refuses: it would start the new taxonomy with most of its edges on its
weakest label, which is the sediment the change exists to remove.
- Write edges with `tools/wikitool xref add --a "<A>" --b "<B>" --rel <label>`, never by
hand. The body region is rendered from `related:`; editing inside a marker pair is
overwritten without warning.
- `cite sync` converts a page's old footnote block into a marked region in passing.
5. **Check each unit mechanically before anything else:**
```bash
tools/wikitool migrate verify --from <pre-migration rev> --path kb/<area> --fail-on-error
```
It compares wikilink and citation **counts**, footnote definitions, H1, structural
frontmatter, and - new in 4.0.0 - the **count of marker pairs per region**. A dropped marker
is otherwise silent: the region becomes ordinary prose and the next write appends a second
one beside it.
6. **Record it, then tighten the checks:**
```bash
tools/wikitool lint # unlabelled_edges and unauthorised_labels must be 0
tools/wikitool migrate done 4.0.0 --pages <N>
```
Only once `lint` reports zero of both is the run finished. The two findings are advisory
during the window and become hard errors afterwards - the same path
`legacy_citation_markers` took after the citation migration.
## How to tell a migrated page from an unmigrated one
Its `related:` entries are `- <label>: <title>` rather than bare titles, and its relationship
and footnote sections sit between `<!-- wikitool:links -->` / `<!-- wikitool:footnotes -->`
marker pairs. `tools/wikitool links show --page "<Title>"` prints `unlabelled` for every edge
still waiting, and `lint`'s `unlabelled_edges` count is the corpus-wide version of the same
question.
## Decision points
- **A label you want is not in the catalogue?** Add it - a row in `link-taxonomy.md` and an
entry in the authorising `COLLECTION.md`. No code change is involved, and the registers are
advisory groupings rather than a schema. Do not stretch a label whose sentence reads false: a
wrong edge is worse than a weak one, because it is machine-readable and will be believed.
- **`related:` holds an entry with no body bullet to derive a label from?** Expected - the
origin repo found 480 edges against 337 bullets, because frontmatter and body had already
drifted apart while the label lived only in prose. Read the page and decide; that drift is
itself part of what this migration repairs.
- **A page loses its last inbound edge?** The orphan check will now report it, and that is the
check working: directional edges mean a page nothing points at is genuinely unreachable, where
the old mirrored model always manufactured a back-link. Either something should point at it,
or it is reached through the catalog and that is fine.
- **Tempted to keep writing reverse edges for navigation?** Do not. `links show` computes the
inbound view, and the rendered bullet on the asserting page is an ordinary `[[wikilink]]`, so
a backlink panel in an editor already shows it.
## Scope
The corpus under `kb/`, plus the three instance-owned declarations in steps 1-3. It does not
touch `raw/`, and it learns nothing new: the same knowledge is restated in a form that can be
checked. Installing the 4.0.0 machinery itself is `INSTALL.md`'s and must have happened first.
+213
View File
@@ -0,0 +1,213 @@
---
type: types/instruction.md
name: private-instance
description: Set up a private working instance as a clone of a public upstream, so stack updates arrive by merge instead of by copying a tarball over the tree.
---
# Set up a private instance against a public upstream
The distribution path in [setup-instance.md](setup-instance.md) builds an instance from a
`dist export` tarball, with no git ancestry in common with the repo it came from. That is the
right shape for someone who only ever *consumes* the stack.
This is the other shape: a private instance that keeps taking stack changes from a public
upstream, and whose own content must never travel back. It costs one safeguard to set up and
saves the whole update procedure afterwards.
**Read this before, not after, the first `publish`.** The gate in step 4 is the thing that makes
the arrangement safe, and adding it later means the window it closes was open in between.
## Why a clone rather than a tarball
`INSTALL.md`'s "Eine Instanz aktualisieren" is `cp -r` as an upgrade strategy: copy `tools/`,
`types/`, `instructions/`, `AGENTS.md`, `VERSION` over the existing tree. It has no three-way
merge, so it cannot notice that the receiving instance changed a file, and it has no conflict
surface, so nobody learns when upstream and local both touched the same one. It overwrites
silently.
A clone gets all of that from git. Stack changes land as real merges, with real conflicts where
they conflict.
**What a plain `git merge` does *not* give you is protection from the upstream's content.** The
private `main` deletes the demo corpus once, but that deletion does not make later upstream
changes to those paths go away. Measured, not assumed:
| Upstream does | `git merge upstream/main` does |
|---|---|
| modifies a page you deleted | `CONFLICT (modify/delete)` - and **leaves the upstream version in your working tree**. Resolve it with `git add -A` and the demo page is back. |
| adds a new page | stages it **silently**. No conflict, no prompt, no mention. |
| deletes a page you also deleted | nothing. The only harmless case. |
The middle row is the one that matters, because nothing announces it. An upstream that ships a
demo corpus *and* uses it as a test bed will add pages, and each one arrives in your instance
and starts showing up in your `lint`, your `index`, your `search` and your `confidence decay`.
So the merge has to be scoped. That is the procedure below, and it is not optional.
## Steps
1. **Clone, and name the two remotes for what they are.**
```bash
git clone <private-repo-url> my-wiki
cd my-wiki
git remote add upstream <public-repo-url>
```
`origin` is yours and is the only thing you ever push to. `upstream` is where stack updates
come from and is fetch-only.
2. **Make the fetch-only half fetch-only in git, too.**
```bash
git remote set-url --push upstream no_push
```
git refuses to push to a URL it cannot resolve. This is a convenience, not the safeguard -
step 4 is the safeguard.
3. **Delete the upstream's demo corpus once, on your own `main`.**
Everything under `kb/` and `raw/` that came with the clone is the upstream's content, not
yours. Remove it with `wikitool rm --page` (never `rm -rf`: `rm` de-links each page from the
rest of the wiki, and a plain delete leaves dead wikilinks and broken citations behind), then
`index rebuild`, `sources rebuild-index`, `lint`.
This is a one-time cut. Afterwards the upstream corpus is frozen from your side, which is
what makes later merges content-free.
4. **Arm the Publish-Remote Gate — before the first `publish`.**
```bash
cat > .wikitool-remotes.json <<'EOF'
{ "schema": 1, "allowed_push_urls": ["<your-private-push-url>"] }
EOF
```
Use the URL `git remote get-url --push origin` prints, exactly. `publish` refuses with exit
42 for anything else, and there is no flag that opens it - see [gates.md](gates.md).
The file is gitignored, so it stays with this checkout and never travels to the upstream.
`wikitool doctor` reports whether the gate is armed, and WARNs at more than one remote
without it.
5. **Take away the write credential, if you can.** A token or deploy key for `origin` only,
with no write access to the upstream, is the one control that holds even if everything above
is misconfigured. Belt and braces.
6. **Personalize and bootstrap.** `USER.md`, `SOUL.md` and optionally `ENVIRONMENT.md` are
yours and unrelated to the upstream's - see the Personalization step of
[setup-instance.md](setup-instance.md), then [bootstrap.md](bootstrap.md) for the venv and
the skills.
A clone inherits the upstream's `kb/CONVENTIONS.md` and `kb/*/COLLECTION.md` rather than
templates, because it inherits the upstream's whole tree. They are yours from this point on:
rewrite them if this instance writes its pages differently - the update procedure below
restores them on every merge, so the change sticks. [kb-profiles.md](kb-profiles.md) has the
alternatives.
## Taking a stack update
Take the machinery, never the content. The merge is held open, the content stages are forced
back to your own state, and only then does it close.
**Three files under those stages are machinery, not content**, and forcing them back is how an
upstream contract change gets silently discarded:
| Path | Why it must take the upstream side |
|---|---|
| `kb/CONTRACT.md` | The stack's own knowledge-layer contract. Every rule in it is enforced by `wikitool`; an instance never edits it |
| `kb/CONVENTIONS.md.template` | The template your `kb/CONVENTIONS.md` was filled from. The filled file is yours; the template is the stack's |
| `raw/CONTRACT.md` | The raw stage's contract, for the same reason as the first row |
Everything else under `kb/` and `raw/` is yours, `kb/CONVENTIONS.md` and each
`kb/<name>/COLLECTION.md` included - they bind your corpus, and they are exactly what the
restore below is protecting.
```bash
BEFORE=$(git rev-parse HEAD)
git fetch upstream
# --no-commit holds the merge open; it may report conflicts under kb/ or raw/,
# which the next four lines are about to make irrelevant.
git merge --no-commit --no-ff upstream/main || true
# Whatever the merge did to the content stages, undo it. HEAD is still your
# pre-merge commit while the merge is open, so this restores exactly your side.
git rm -rq --cached --ignore-unmatch kb raw
rm -rf kb raw
git checkout HEAD -- kb raw
# ...then take the upstream side back for the machinery that lives among it.
# MERGE_HEAD is still resolvable while the merge is open.
git checkout MERGE_HEAD -- kb/CONTRACT.md kb/CONVENTIONS.md.template raw/CONTRACT.md
git commit --no-edit
```
Then **check that it worked**, rather than trusting that it did. The same three paths are
excluded here, spelled out rather than held in a variable so that the check can be read on its
own and copied on its own:
```bash
git diff --name-only "$BEFORE" HEAD -- kb raw \
| grep -vE '^(kb/CONTRACT\.md|kb/CONVENTIONS\.md\.template|raw/CONTRACT\.md)$'
```
Must print nothing.
An empty result is the proof that the update touched machinery only. A non-empty one means a
path slipped through - inspect it before going further.
**The exclusion is not cosmetic.** Without it the check reports *empty* for an update that just
ate a `kb/CONTRACT.md` change - it would be confirming the failure it exists to catch. If one of
the three paths does not appear in the diff at all, that is fine: it means upstream did not
touch it.
Then, as after any stack change: `doctor`, `docs verify`, `instructions verify`, `migrate status`,
`lint`. A `migrate status` with outstanding links means the update crossed a compatibility
boundary - follow [migrate-corpus.md](migrate-corpus.md) before doing anything else.
**Why not just `git merge upstream/main`?** Because of the table above: a page the upstream
*adds* arrives with no conflict and no message. You would find out when `lint` starts reporting
pages you never wrote - if you noticed at all.
## Where stack development happens
**In the public repo, not here.** That is not a preference; the stack is built that way. The
development-only half of the instruction layer is pruned from a distribution one-way, with no
command that reconstructs it, so an instance built this way has no tool-development mode to
switch into in the first place.
When a tool bug blocks real content work here - and it will - file the issue against the public
repo (an MCP server or the web UI reaches it from any session; no shared history needed), fix it
there where the tests, `docs verify` and CI's version gate live, and take the fix back with the
merge above. Nothing is lost by the detour: the fix has to pass that CI either way.
## Decision points
- **Merge conflict in `kb/` or `raw/`?** Expected, and already handled: the update procedure
above overwrites those stages with your own afterwards, so the conflict resolves itself.
Never resolve one by hand with `git add -A` - that is exactly how the upstream version, which
git left sitting in your working tree, gets committed into your instance.
- **`git diff` after the merge shows something under `kb/` or `raw/`?** Stop - unless it is one
of the three machinery paths the check excludes, which is the update working as intended. For
anything else the scoping step did not take: do not publish; find out which path came through
and where from.
- **Conflict in `tools/`, `types/` or `instructions/`?** You changed the stack locally, which
step "Where stack development happens" says not to do. Take the upstream side and re-file the
change as an issue there.
- **...but you changed how *your pages* are written?** That is not a stack change and the rule
above does not apply to it. Language, section headings, naming forms, tone, relationship
labels and the confidence rubric live in `kb/CONVENTIONS.md`, and each collection's authoring
rules in `kb/<name>/COLLECTION.md` - all under `kb/`, all yours, all restored by the merge
procedure rather than overwritten by it. If you find yourself editing `tools/` or `types/` to
change an authoring convention, that is a stack bug: file it, because the split exists
precisely so you do not have to.
## Scope
Not for a first instance with no upstream - that is [setup-instance.md](setup-instance.md). Not
for a fresh clone of a repo you already own and develop in - that is
[bootstrap.md](bootstrap.md). This is specifically the two-remote case, where the cost of a
mistaken push is disclosure rather than inconvenience.
+63 -16
View File
@@ -60,24 +60,71 @@ bereit für den ersten `Ingest`.
- Nicht genannt: lokal bleiben - dann braucht **jeder** spätere `tools/wikitool publish` - Nicht genannt: lokal bleiben - dann braucht **jeder** spätere `tools/wikitool publish`
ein `--no-push` (dessen Branch-Prüfung dabei ohnehin entfällt, siehe Schritt 2). ein `--no-push` (dessen Branch-Prüfung dabei ohnehin entfällt, siehe Schritt 2).
5. **Entscheidungspunkt - KB-Sprache.** Frage den Nutzer, in welcher Sprache die Seiten unter 5. **Entscheidungspunkt - Autorenkonventionen.** Die Distribution bringt keine ausgefüllten
`kb/` geschrieben werden sollen. Diese Instanz erbt aus dem Quell-Repo **Deutsch** - sowohl die Konventionen mit, sondern `kb/CONVENTIONS.md.template` und je Collection ein
Regel in [kb/CONTRACT.md](../kb/CONTRACT.md#language) als auch das Vokabular in `kb/<name>/COLLECTION.md.template`. Beide **binden**, sobald sie übernommen sind, und beide
[german-terminology.md](german-terminology.md) und die deutschen Abschnittsnamen in gehören dieser Instanz - deshalb liefert der Stack nur die Vorlage. Die eine Entscheidung
`tools/chemenu/sections.py`. Das ist eine Entscheidung der Ursprungsinstanz, keine dahinter ist: **in welcher Sprache und in welchem Ton schreibt diese Instanz ihre Seiten?**
Eigenschaft des Musters, und sie wird hier nicht stillschweigend weitergereicht.
- **Deutsch bestätigt:** nichts zu tun. Ablauf:
- **Andere Sprache:** *vor dem ersten Ingest* umstellen, denn danach ist es eine Migration
jeder vorhandenen Seite. Zu ändern sind der Abschnitt "Language" in `kb/CONTRACT.md`, die
Tonfall-Beispiele und Hedge-Wörter darunter, die vier Page-Type-Templates in `types/`, die
kanonischen Namen in `sections.py` (die bisherigen als Alias behalten) und die
Beziehungslabels in `kb/CONTRACT.md` § Linking. `german-terminology.md` wird dann ersetzt
oder gelöscht.
Unverändert bleibt in jedem Fall die eigentliche Regel: **jede Zeile einer Seite ist Prosa 1. Die Collection-Contracts **und die Page-Type-Specs** übernehmen - Kopien, keine Frage an
oder Identifier, und nur Prosa wird übersetzt.** Titel, Wikilink-Ziele, Cite-IDs, Enum-Werte, den Nutzer, denn was dort steht ist als Ausgangspunkt unabhängig von der Sprache brauchbar:
Tags, Befehle und Pfade folgen keiner KB-Sprache.
```bash
for template in kb/*/COLLECTION.md.template types/*.template; do
cp "$template" "${template%.template}"
done
```
Die `.template`-Dateien bleiben liegen; sie sind die Vorlage für den nächsten Export.
Unter `types/` betrifft das genau die Type-Specs mit `root: kb` - `entity`, `concept`,
`source`, `comparison` - samt ihrer `.schema.yaml`. Sie beschreiben Seiten, die *diese*
Instanz schreibt, also gehören sie ihr: Prosa, Template und Sprache dürfen umgeschrieben
werden. `instruction`, `lint-report` und `type-spec` beschreiben Stack-Artefakte und
kommen unverändert.
2. Den Nutzer nach der KB-Sprache fragen. `kb/CONVENTIONS.md.template` ist auf **Englisch**
voreingestellt; [kb-profiles.md](kb-profiles.md) hält daneben ein vollständiges
deutsches Profil bereit, und dessen Volltext ist die `kb/CONVENTIONS.md` des Quell-Repos.
Der Profilkatalog ist eine **Palette, kein Enum**: übernommen wird der Text *in* die
Instanzdatei, nicht ein Verweis auf den Katalog.
3. `kb/CONVENTIONS.md.template` nach `kb/CONVENTIONS.md` kopieren, entlang des gewählten
Profils ausfüllen - Sprache, Abschnittsnamen, Namensformen, Ton, Beziehungslabels,
Confidence-Rubrik - und dabei die Sentinel-Zeile (`wikitool:template-unfilled`) entfernen.
Die Platzhalter in geschweiften Klammern **sind** der Fragenkatalog.
4. Bei einer anderen Sprache als der des Quell-Repos: `german-terminology.md` löschen oder
durch das eigene Vokabular ersetzen - sie ist Material des deutschen Profils, nicht des
Stacks.
**Vor dem ersten Ingest entscheiden.** Die `sections:`-Namen in `kb/CONVENTIONS.md` sind die
Überschriften, die `xref` und `cite` in jede Seite schreiben; sie danach zu ändern ist eine
Migration jeder vorhandenen Seite (`section_aliases:` trägt die alten Namen, siehe
[migrate-corpus.md](migrate-corpus.md)).
**Nichts davon liegt in einer Stack-Datei.** Der Compiler liest die Abschnittsnamen aus
`kb/CONVENTIONS.md`; die vier Page-Type-Specs gehören ab Schritt 1 dieser Instanz. Eine
anderssprachige Instanz übersetzt sie einfach - das ist kein lokaler Patch an etwas
Ausgeliefertem mehr, sondern Arbeit an den eigenen Dateien, und ein Upgrade nimmt sie ihr
nicht wieder weg.
Was der Stack von `types/` überhaupt noch verlangt, ist eine Zeile: es muss einen Type-Spec
mit `name: source` geben, dessen Schema `raw_files` fordert. Daran hängt der gesamte
`raw/`→`kb/`-Provenance-Pfad (`sources coverage`, `[^cite-id]`-Auflösung, `kb/provenance.md`),
und `docs verify` prüft genau das - nicht mehr.
Unverändert bleibt in jedem Fall die Regel, die dem Stack gehört: **jede Zeile einer Seite
ist Prosa oder Identifier, und nur Prosa wird übersetzt** ([kb/CONTRACT.md § Language and
identifiers](../kb/CONTRACT.md#language-and-identifiers)). Titel, Wikilink-Ziele, Cite-IDs,
Enum-Werte, Tags, Befehle und Pfade folgen keiner KB-Sprache.
`tools/wikitool doctor` prüft das Ergebnis in Schritt 12 (`conventions`): eine fehlende
Datei ist ein `FAIL`, eine mit Sentinel oder ohne vollständigen `sections:`-Block ebenso.
`docs verify` prüft zusätzlich `profile:` und `required_by_stack:` auf jedem
`COLLECTION.md`.
6. **Entscheidungspunkt - Personalization.** Die Distribution bringt 6. **Entscheidungspunkt - Personalization.** Die Distribution bringt
`USER.md.template` und `SOUL.md.template` mit, aber keine ausgefüllten Fassungen: wer diese `USER.md.template` und `SOUL.md.template` mit, aber keine ausgefüllten Fassungen: wer diese
+6 -4
View File
@@ -61,8 +61,9 @@ pages should never have cost the concept contract. Field-level requirements alwa
article also pass `--set source_url=<upstream URL>`; `raw_files:` must still point at the article also pass `--set source_url=<upstream URL>`; `raw_files:` must still point at the
local copy. Then write the Summary / Key Takeaways / Action Items prose from step 4 - in the local copy. Then write the Summary / Key Takeaways / Action Items prose from step 4 - in the
KB language, whatever the source's own language is, quoting verbatim passages in the KB language, whatever the source's own language is, quoting verbatim passages in the
original. The rule and what is exempt from it: original. Which language that is: [kb/CONVENTIONS.md](../../kb/CONVENTIONS.md#language).
[kb/CONTRACT.md](../../kb/CONTRACT.md#language). What is exempt from it, in any language:
[kb/CONTRACT.md](../../kb/CONTRACT.md#language-and-identifiers).
Fill `## Not Extracted` in the same pass: what you read and deliberately did not promote, Fill `## Not Extracted` in the same pass: what you read and deliberately did not promote,
with the reason. Nothing in the repository can re-derive that judgment, and without it the with the reason. Nothing in the repository can re-derive that judgment, and without it the
@@ -70,8 +71,9 @@ pages should never have cost the concept contract. Field-level requirements alwa
6. **Create or update entity pages.** Read 6. **Create or update entity pages.** Read
[kb/entities/COLLECTION.md](../../kb/entities/COLLECTION.md) and [kb/entities/COLLECTION.md](../../kb/entities/COLLECTION.md) and
[kb/CONTRACT.md](../../kb/CONTRACT.md) first - the second is where tone, naming, provenance [kb/CONTRACT.md](../../kb/CONTRACT.md) plus
and citation are defined. [kb/CONVENTIONS.md](../../kb/CONVENTIONS.md) first - the second is where provenance and
citation are defined, the third where this instance's tone and naming forms are.
New: New:
+7 -5
View File
@@ -13,10 +13,12 @@ integrating into an existing one.
**Before the first `wikitool` call:** [session-setup.md](../session-setup.md). **Before the first `wikitool` call:** [session-setup.md](../session-setup.md).
**Read before drafting:** [kb/CONTRACT.md](../../kb/CONTRACT.md) - naming, tone, linking, **Read before drafting:** [kb/CONTRACT.md](../../kb/CONTRACT.md) - linking, provenance and the
provenance and confidence - together with the target collection's own `COLLECTION.md`, which confidence machinery, all of which the tool enforces - and
carries its quality goal and what is local to that subtree. Field-level requirements come from [kb/CONVENTIONS.md](../../kb/CONVENTIONS.md), which is where this instance's language, naming
`tools/wikitool types describe <type>`. forms, tone and relationship labels are, together with the target collection's own
`COLLECTION.md`, which carries its quality goal and what is local to that subtree. Field-level
requirements come from `tools/wikitool types describe <type>`.
## Creating a page ## Creating a page
@@ -46,7 +48,7 @@ carries its quality goal and what is local to that subtree. Field-level requirem
subjects - so the prose connects to existing pages instead of restating them. subjects - so the prose connects to existing pages instead of restating them.
5. **Draft.** Fill in the generated skeleton's TODO sections, following the tone rules in 5. **Draft.** Fill in the generated skeleton's TODO sections, following the tone rules in
[kb/CONTRACT.md](../../kb/CONTRACT.md#tone). If `provenance:` is `sourced` or `mixed`, cite [kb/CONVENTIONS.md](../../kb/CONVENTIONS.md#tone). If `provenance:` is `sourced` or `mixed`, cite
hard facts as you write them with `tools/wikitool cite add --page "<Title>" --source hard facts as you write them with `tools/wikitool cite add --page "<Title>" --source
"Source - X"`, which also adds `X` to `sources:` - paste the `[^cite-id]` marker it prints. "Source - X"`, which also adds `X` to `sources:` - paste the `[^cite-id]` marker it prints.
+110 -82
View File
@@ -7,12 +7,24 @@ material in `raw/`, and is expected to stay correct without being re-derived.
**Quality goal:** a page should answer a future question *without* re-reading the source it **Quality goal:** a page should answer a future question *without* re-reading the source it
came from. If answering still requires the raw file, the page is incomplete. came from. If answering still requires the raw file, the page is incomplete.
This file holds the rules that apply in **every** collection. Each `kb/<name>/COLLECTION.md` This file holds the rules that apply in **every** collection **and in every instance**. That
declares that it inherits them and adds only what is local to its own subtree - read this file second half is the cut: what is written here is enforced by `tools/wikitool` or follows from
together with the target collection's contract before writing or editing a page. how it works, so it is identical everywhere and `dist export` ships it verbatim.
**What an instance decides for itself is next door, in
[kb/CONVENTIONS.md](CONVENTIONS.md)** - the language pages are written in, the headings its two
generated regions render under, the naming forms, the tone, the confidence rubric. That file binds exactly as this one does; it is simply owned by the instance
rather than by the stack, so the distribution ships only its `.template` and the instance writes
the real one. Read both, plus the target collection's `kb/<name>/COLLECTION.md` (also
instance-owned), before writing or editing a page.
The split is by **who may change the sentence**, not by what it is about. Language, tone and
naming used to sit here, which meant every instance that answered "not German" to
`setup-instance.md` was locally editing a file the stack also ships - and a merge from upstream
would quietly hand it back.
Structural facts (which frontmatter fields exist, which are required, what the body skeleton Structural facts (which frontmatter fields exist, which are required, what the body skeleton
looks like) are *not* here - they belong to the type-specs and are printed by looks like) are in neither - they belong to the type-specs and are printed by
`tools/wikitool types describe <type>`. Never hand-write frontmatter; scaffold with `tools/wikitool types describe <type>`. Never hand-write frontmatter; scaffold with
`tools/wikitool new <type> --name "<Name>" --set field=value ...`. `tools/wikitool new <type> --name "<Name>" --set field=value ...`.
@@ -21,7 +33,15 @@ looks like) are *not* here - they belong to the type-specs and are printed by
`kb/` is a **namespace, not a collection**. It carries no `COLLECTION.md` of its own. `kb/` is a **namespace, not a collection**. It carries no `COLLECTION.md` of its own.
A directory under `kb/` is a **collection** exactly when it contains a `COLLECTION.md`. That A directory under `kb/` is a **collection** exactly when it contains a `COLLECTION.md`. That
file is the local authoring contract for every page in the subtree. file is the local authoring contract for every page in the subtree, and it belongs to the
instance: it declares in its frontmatter which profile from
[instructions/kb-profiles.md](../instructions/kb-profiles.md) it adopted, and whether the stack
resolves against it by name.
| Field | Means |
|---|---|
| `profile:` | Which catalogue entry this contract started from, or `none`. Free text - the catalogue is a palette, not an enum, and a collection an instance invented has no entry to name |
| `required_by_stack:` | Whether `wikitool` itself depends on this collection *by name*. Not the instance's to choose: `docs verify` checks it against the stack's own list. `kb/sources/` is `true` - `sources coverage`, `[^cite-id]` resolution and `kb/provenance.md` all resolve against that name - and everything else is `false` |
- A subdirectory *inside* a collection is an **area**. It inherits the enclosing contract and - A subdirectory *inside* a collection is an **area**. It inherits the enclosing contract and
must not carry a `COLLECTION.md` of its own - `kb/entities/systems/` is an area of must not carry a `COLLECTION.md` of its own - `kb/entities/systems/` is an area of
@@ -40,9 +60,11 @@ file is the local authoring contract for every page in the subtree.
| `kb/sources/` | One summary page per ingested source, carrying its `raw_files:` provenance | [sources/COLLECTION.md](sources/COLLECTION.md) | | `kb/sources/` | One summary page per ingested source, carrying its `raw_files:` provenance | [sources/COLLECTION.md](sources/COLLECTION.md) |
| `kb/comparisons/` | Structured comparisons of two or more existing pages | [comparisons/COLLECTION.md](comparisons/COLLECTION.md) | | `kb/comparisons/` | Structured comparisons of two or more existing pages | [comparisons/COLLECTION.md](comparisons/COLLECTION.md) |
**Adding a collection:** `mkdir kb/<name>` and write a `kb/<name>/COLLECTION.md`. Collections The four rows above are this instance's collections, not a fixed set. **Adding one:**
`mkdir kb/<name>` and write a `kb/<name>/COLLECTION.md` with the two fields above. Collections
are discovered by contract presence, so no code change is needed. A collection only becomes are discovered by contract presence, so no code change is needed. A collection only becomes
*writable* once some type-spec declares a matching `base_dir:`. *writable* once some type-spec declares a matching `base_dir:`. Renaming or dropping one is the
instance's call too - except where `required_by_stack: true` says otherwise.
**Where a page goes** is decided by its type-spec, never by hand - see **Where a page goes** is decided by its type-spec, never by hand - see
[types/type-spec.md](../types/type-spec.md). [types/type-spec.md](../types/type-spec.md).
@@ -61,53 +83,41 @@ Never hand-edit these; they are produced by `tools/wikitool`:
To *find* a page, search rather than read the catalog: `tools/wikitool search "<text>"`, or To *find* a page, search rather than read the catalog: `tools/wikitool search "<text>"`, or
`tools/wikitool search --field <predicate>` for a structured query over frontmatter. `tools/wikitool search --field <predicate>` for a structured query over frontmatter.
## Naming ## Titles are identifiers
- Human-readable titles with spaces: `Hybrid Search.md`, `Gitea Actions.md` - not kebab-case. **The filename stem *is* the page title, and `[[wikilinks]]` must match it exactly.** That is
- Singular for entities: `ha-core.md`, not `ha-cores.md`. not a naming preference; it is the wiki's only way to address a page. `wikitool lint` reports an
- Comparison pages read as a comparison: `Go vs Rust.md`. H1 that stops matching its title, `rename`/`rm` rewrite every reference to a stem, and a
- ADRs are prefixed: `adr-001-use-go-modules.md`. `[^cite-id]` resolves through one.
- The filename stem *is* the page title, and `[[wikilinks]]` must match it exactly.
- Prefer readability over convention when the two conflict.
What to name a thing: projects use their repository or common name; systems a descriptive Which *form* those titles take - spaces or kebab-case, singular or plural, what prefixes a
name; tools the tool's own name; technologies their standard spelling and capitalization; decision record - is the instance's, in
people a full name or common handle. [kb/CONVENTIONS.md § Naming](CONVENTIONS.md#naming).
## Every page should ## Every page should
- [ ] Carry a clear, descriptive title and a summary near the top - [ ] Carry a clear, descriptive title and a summary near the top
- [ ] Use consistent terminology with the rest of the wiki - [ ] Use consistent terminology with the rest of the wiki
- [ ] Link to every entity and concept it mentions, and be linked to in return - [ ] Link to the entities and concepts it mentions, and declare an edge where the relationship
is worth naming - in the direction this page asserts it, not in both
- [ ] Cite its hard facts (see [Provenance and citation](#provenance-and-citation)) - [ ] Cite its hard facts (see [Provenance and citation](#provenance-and-citation))
- [ ] Duplicate no existing page - [ ] Duplicate no existing page
- [ ] Appear in the catalog (guaranteed by `wikitool index rebuild`) - [ ] Appear in the catalog (guaranteed by `wikitool index rebuild`)
## Tone ## Quotation cap
Wikipedia style: factual, neutral, specific. At most 2 blockquoted lines per page. `wikitool lint` reports overages as advisory, since
exceeding the cap can be a legitimate judgment call - but the page should carry the knowledge
itself, not delegate it to quotations. The cap is about how much of the page you let quotes
carry; it does not apply to text you are citing verbatim from a source.
- No buzzwords ("bahnbrechend", "hochmodern", "leistungsstark", "revolutioniert"). The register those lines are written in - what counts as a buzzword, what filler is refused -
- No AI filler ("es sei angemerkt", "es ist wichtig zu betonen", "in der heutigen Zeit"). is the instance's, in [kb/CONVENTIONS.md § Tone](CONVENTIONS.md#tone).
- No em-dash asides carrying parenthetical reasoning.
- At most 2 blockquoted lines per page. `wikitool lint` reports overages as advisory, since
exceeding the cap can be a legitimate judgment call - but the page should carry the
knowledge itself, not delegate it to quotations. The cap is about how much of the page you
let quotes carry; it does not apply to text you are citing verbatim from a source.
Good: "MQTT ist ein leichtgewichtiges Publish-Subscribe-Protokoll für Geräte mit knappen ## Language and identifiers
Ressourcen."
Bad: "MQTT ist ein bahnbrechendes, hochmodernes Protokoll, das die IoT-Kommunikation *Which* language pages are written in is [kb/CONVENTIONS.md](CONVENTIONS.md)'s to say. What
revolutioniert - und es sei angemerkt, dass es ein Publish-Subscribe-Muster verwendet." follows here is the part that is not a choice, because the tool resolves against it.
## Language
Pages are written in **German**. This binds `kb/` and the authoring surface that shapes it -
the page type-specs `types/entity.md`, `types/concept.md`, `types/source.md` and
`types/comparison.md`. `raw/` is untouched ([raw/CONTRACT.md](../raw/CONTRACT.md)), and the
control plane stays English: AGENTS.md, the stage contracts including this one, `instructions/`,
and the type-specs for non-page artifacts.
Every line of a page is either **prose** or an **identifier**. Only prose is translated. Every line of a page is either **prose** or an **identifier**. Only prose is translated.
@@ -118,55 +128,75 @@ source page's Summary / Key Takeaways / Action Items / Not Extracted, and `summa
| Identifier | Why | | Identifier | Why |
|---|---| |---|---|
| Page titles, and the H1 that repeats one | A title is the wiki's only identifier for a page and follows the subject's own established name - see [Naming](#naming). `wikitool lint` reports an H1 that stops matching its title | | Page titles, and the H1 that repeats one | A title is the wiki's only identifier for a page and follows the subject's own established name - see [Titles are identifiers](#titles-are-identifiers). `wikitool lint` reports an H1 that stops matching its title |
| The subtype value on the generated `**Typ:**` line | It renders a schema enum value (`technology`, `workflow`), which `search --field` filters on. The label is prose; the value is not | | The subtype value on the generated `**Typ:**` line | It renders a schema enum value (`technology`, `workflow`), which `search --field` filters on. The label is prose; the value is not |
| `tags:` | Search keys, not prose | | `tags:` | Search keys, not prose |
| Commands, paths, config keys, hostnames, code | They are what they are | | Commands, paths, config keys, hostnames, code | They are what they are |
| Quotations | Quoted verbatim in the source's own language | | Quotations | Quoted verbatim in the source's own language |
Established English technical terms stay English inside German prose - "GitOps", "Ownership Which foreign technical terms stay untranslated inside that prose is a judgment call the
Model", "Reverse Proxy", "Pull Request". Translate a term only where the German one is genuinely instance records - see [kb/CONVENTIONS.md § Language](CONVENTIONS.md#language).
the more common usage. A coined German equivalent nobody else writes makes the page harder to
find, not more idiomatic.
Which terms those are, which have a settled German form, and the register the prose is written in:
[instructions/german-terminology.md](../instructions/german-terminology.md). It is lookup material,
not a second rule - every entry in it is a decision that was made wrong once first.
**A source in another language** is still summarized in the KB language: a source page is **A source in another language** is still summarized in the KB language: a source page is
evidence *about* a source, not a substitute for it. Quote verbatim in the original language and evidence *about* a source, not a substitute for it. Quote verbatim in the original language and
record the raw file's language in `source_language:`. record the raw file's language in `source_language:`.
### Section headings ### Generated regions
Three headings are a vocabulary the tool owns rather than prose an author picks: `xref add` Two regions of a page body are **generated**, not authored: the links region `xref` owns and the
writes into Relationships and See Also, and `cite add` owns the trailing Footnotes block. They footnotes region `cite` owns. Each sits between a marker pair:
follow the KB language like everything else - `## Beziehungen`, `## Siehe auch`, `## Fußnoten` -
and `tools/chemenu/sections.py` is the single place naming them.
Each has aliases the tool still *recognizes* but no longer writes, which is what lets the corpus ```markdown
be translated page by page: a page still carrying `## Relationships` is found and appended to <!-- wikitool:links -->
correctly, and `cite sync` leaves an untranslated `## Footnotes` heading alone rather than ## Beziehungen
retitling it. Renaming a heading is the translation pass's job, never a side effect of another
command. Any *other* heading an author adds is ordinary prose and is translated with the rest. - **depends-on:** [[Hermes]]
<!-- /wikitool:links -->
```
The marker is what the tool locates the region by, and everything between the markers -
**heading included** - is replaced wholesale on the next write. An author never edits inside
them; anything left there is overwritten without warning, exactly as in `kb/index.md`. A region
with nothing to show is absent rather than empty.
The heading is therefore a *rendering* value, taken from `kb/CONVENTIONS.md`'s `sections:`. No
heading text exists in the compiler, and nothing matches on it: changing the declaration
re-renders the words on the next write and cannot split a page.
That is not how it used to work. The tool located these regions by matching their heading text,
which made a translated heading a structural fact - and made the region's *end* a guess. It ran
to the next heading, and before that to the end of the file, which silently deleted whatever sat
after it on eight pages. Any *other* heading a page carries is ordinary prose.
## Linking ## Linking
Every page links to what it mentions, in both directions. Cross-references are created with **An edge is authored in one direction**, on the page that asserts it, and carries a label that
`tools/wikitool xref add --a "<A>" --b "<B>" --rel-a "<label>" --rel-b "<label>"`, never by is a machine value rather than prose:
hand-editing the `related:` array or the Relationships/See Also bullets.
Use a typed relationship label rather than a generic one: ```yaml
related:
- depends-on: Hermes
```
`hängt ab von` · `verwendet` · `implementiert` · `erweitert` · `ersetzt` · `steht in Konflikt mit` Created with `tools/wikitool xref add --a "<A>" --b "<B>" --rel <label>`, never by hand-editing
· `benötigt` · `erzeugt` · `konsumiert` · `besitzt` · `pflegt` · `läuft auf` · `verwandt mit` `related:` or the rendered bullet. Say the sentence before choosing the label - `[A] <label>
(last resort) [B]` - and if it only reads true backwards, the edge belongs on the other page.
The labels are prose written into a `- **label:** [[Title]]` bullet; no code matches on them, so **A reverse edge is a separate decision, not a mirror.** Write one when it independently helps a
an untranslated page's English label is stale wording, not a broken reference. reader at the other end; do not write one to make the graph symmetric. Navigation does not
depend on it either way: `index rebuild` renders the inbound view from the graph, completely and
without maintenance.
A page is expected to have at least one inbound link; `wikitool lint` reports orphans. Which labels exist is [instructions/link-taxonomy.md](../instructions/link-taxonomy.md), a
Comparison pages are exempt - they are reached through the catalog. palette that binds nothing. Which of them a page may *use* is its own collection's `outbound:`
block, per destination - the **source** collection decides, because the rules that govern an
edge are the rules of the collection asserting it. `xref add` refuses an unauthorised label and
`lint` reports one.
A page is expected to have at least one inbound edge; `wikitool lint` reports orphans.
Comparison pages are exempt - they are reached through the catalog. Directional edges mean more
pages qualify than under the old mirrored model, and that is the check measuring reachability
rather than measuring whether `xref` ran.
Renaming a page, deleting one, or dropping a single reference are tool operations with their Renaming a page, deleting one, or dropping a single reference are tool operations with their
own procedure: see [instructions/page-lifecycle.md](../instructions/page-lifecycle.md). own procedure: see [instructions/page-lifecycle.md](../instructions/page-lifecycle.md).
@@ -187,15 +217,16 @@ Every claim is either traceable to a raw file or explicitly marked as not.
command, or config value. `tools/wikitool cite add --page "<Title>" --source "Source - X" command, or config value. `tools/wikitool cite add --page "<Title>" --source "Source - X"
[--file <qualifier>]` mints the id, upserts its `[[Source - X]]` (or [--file <qualifier>]` mints the id, upserts its `[[Source - X]]` (or
`[[Source - X|storage-model.md]]` for a multi-file source) definition in the page's trailing `[[Source - X|storage-model.md]]` for a multi-file source) definition in the page's trailing
`## Footnotes` block, and adds `Source - X` to `sources:` - it prints the marker to paste at Footnotes block (named per [Section headings](#section-headings)), and adds `Source - X` to
`sources:` - it prints the marker to paste at
the fact; placing it is still manual. Never hand-type a cite-id (AGENTS.md invariant 1). This the fact; placing it is still manual. Never hand-type a cite-id (AGENTS.md invariant 1). This
differs from a plain `[[Source - X]]` link, which only means "related to". differs from a plain `[[Source - X]]` link, which only means "related to".
- **Notation inside code is notation, not a reference.** A `[^cite-id]` or a `[[wikilink]]` - **Notation inside code is notation, not a reference.** A `[^cite-id]` or a `[[wikilink]]`
written in backticks or a fenced block is read as an example: the citation does not count and written in backticks or a fenced block is read as an example: the citation does not count and
the link does not exist. That is what lets a page document this stack's own syntax. It also the link does not exist. That is what lets a page document this stack's own syntax. It also
means a marker appended to a line *inside* a fence cites nothing - put it on a means a marker appended to a line *inside* a fence cites nothing - put it on a source line
`Quelle: [^cite-id]` line under the block, where it renders as a footnote instead of under the block (`<source-word>: [^cite-id]`, in the KB language), where it renders as a
travelling with the command when someone copies it. footnote instead of travelling with the command when someone copies it.
- A source cited inline must also appear in the page's frontmatter `sources:` list; - A source cited inline must also appear in the page's frontmatter `sources:` list;
`wikitool lint` checks this in both directions, and hard-errors on a leftover pre-migration `wikitool lint` checks this in both directions, and hard-errors on a leftover pre-migration
`^[[...]]` marker, an undefined `[^cite-id]` reference, or an orphaned Footnotes definition. `^[[...]]` marker, an undefined `[^cite-id]` reference, or an orphaned Footnotes definition.
@@ -215,23 +246,20 @@ one - and never file the synthesized version back into the wiki.
`confidence` is *derived* from it by `tools/wikitool confidence decay` and must never be `confidence` is *derived* from it by `tools/wikitool confidence decay` and must never be
edited directly. edited directly.
Base score for a single source is 0.5, adjusted by:
- **+0.2 per supporting source** (max +0.6)
- **+0.2** if confirmed <30 days ago, **+0.1** if <90 days
- **+0.1** for official documentation, **+0.05** for a reputable secondary source
- **+0.1** if multiple independent sources agree
Re-assess a page with `tools/wikitool touch --page "<Title>" --confidence-base <value>`. Re-assess a page with `tools/wikitool touch --page "<Title>" --confidence-base <value>`.
In prose, hedge according to the score: below 0.6 write "möglicherweise"/"kann"; below 0.4 What the number *means* - the base score, what raises it and by how much, and how to hedge in
write "unsicher"/"unbestätigt". prose below a threshold - is a rubric rather than a mechanism, so it is
[kb/CONVENTIONS.md § Confidence rubric](CONVENTIONS.md#confidence-rubric)'s.
## What does not belong here ## What does not belong here
- Raw source material - it stays immutable under `raw/`. - Raw source material - it stays immutable under `raw/`.
- Type definitions, frontmatter contracts, or templates - those live in `types/`. - Type definitions, frontmatter contracts, or templates - those live in `types/`.
- Procedures for operating the tooling - those live in `instructions/`. - Procedures for operating the tooling - those live in `instructions/`.
- **Anything an instance would have to rewrite for itself** - language, naming forms, tone,
relationship labels, the confidence rubric. Those are `kb/CONVENTIONS.md`'s, and a sentence
of that kind here is a sentence the stack ships over the instance's own answer.
- Rules that apply to only one collection - those belong in that collection's - Rules that apply to only one collection - those belong in that collection's
`COLLECTION.md`. `COLLECTION.md`.
- Hand-edited generated files - see [Generated files](#generated-files). - Hand-edited generated files - see [Generated files](#generated-files).
+119
View File
@@ -0,0 +1,119 @@
---
language: de
profile: german
sections:
links: Beziehungen
footnotes: Fußnoten
---
# kb/ - Authoring Conventions of This Instance
The decisions [kb/CONTRACT.md](CONTRACT.md) deliberately does not make. The contract holds what
the code enforces and is identical in every instance; this file holds what *this* instance
chose, and no other instance has to agree with a word of it.
**It binds all the same.** Everything below applies to every page under `kb/`, exactly as the
contract does. The difference is ownership, not authority: a rule here is changed by editing
this file, a rule there by changing the stack.
Adopted from the `german` profile in
[instructions/kb-profiles.md](../instructions/kb-profiles.md). That catalogue is a palette, not
an enum - what is written here is what holds, whether or not a profile says the same thing.
The frontmatter above is the one machine-read part. `sections:` names the headings the two
**generated regions** render under - the links region `wikitool xref` owns and the footnotes
region `wikitool cite` owns. Each sits between a marker pair, and the marker is what the tool
locates it by, so the heading here is a display value: changing it re-renders the words above
those regions and nothing else. Nothing matches on this text.
## Language
Pages are written in **German**. This binds `kb/` and the authoring surface that shapes it -
the page type-specs `types/entity.md`, `types/concept.md`, `types/source.md` and
`types/comparison.md`. `raw/` is untouched ([raw/CONTRACT.md](../raw/CONTRACT.md)), and the
control plane stays English: `AGENTS.md`, the stage contracts, this file, `instructions/`, and
the type-specs for non-page artifacts.
Which line is prose and which is an identifier - and therefore what is translated at all - is
the contract's rule, not this file's: see
[kb/CONTRACT.md § Language and identifiers](CONTRACT.md#language-and-identifiers).
Established English technical terms stay English inside German prose - "GitOps", "Ownership
Model", "Reverse Proxy", "Pull Request". Translate a term only where the German one is genuinely
the more common usage. A coined German equivalent nobody else writes makes the page harder to
find, not more idiomatic.
Which terms those are, which have a settled German form, and the register the prose is written
in: [instructions/german-terminology.md](../instructions/german-terminology.md). It is lookup
material, not a second rule - every entry in it is a decision that was made wrong once first.
### Section headings
The two generated regions render under `## Beziehungen` and `## Fußnoten`. An author never
writes inside them - they are rebuilt from frontmatter on every write, exactly like
`kb/index.md` - and never has to write the heading either. Any *other* heading on a page is
ordinary prose and is translated with the rest.
There is no `## Siehe auch` region any more. It was the reciprocal half of a bidirectional
`xref add`; under authored directional edges, `see-also` is a *label* inside the links region.
## Naming
- Human-readable titles with spaces: `Hybrid Search.md`, `Gitea Actions.md` - not kebab-case.
- Singular for entities: `ha-core.md`, not `ha-cores.md`.
- Comparison pages read as a comparison: `Go vs Rust.md`.
- ADRs are prefixed: `adr-001-use-go-modules.md`.
- Prefer readability over convention when the two conflict.
What to name a thing: projects use their repository or common name; systems a descriptive
name; tools the tool's own name; technologies their standard spelling and capitalization;
people a full name or common handle.
The one naming fact that is *not* a choice, and therefore lives in the contract: the filename
stem is the page title, and `[[wikilinks]]` must match it exactly.
## Tone
Wikipedia style: factual, neutral, specific.
- No buzzwords ("bahnbrechend", "hochmodern", "leistungsstark", "revolutioniert").
- No AI filler ("es sei angemerkt", "es ist wichtig zu betonen", "in der heutigen Zeit").
- No em-dash asides carrying parenthetical reasoning.
Good: "MQTT ist ein leichtgewichtiges Publish-Subscribe-Protokoll für Geräte mit knappen
Ressourcen."
Bad: "MQTT ist ein bahnbrechendes, hochmodernes Protokoll, das die IoT-Kommunikation
revolutioniert - und es sei angemerkt, dass es ein Publish-Subscribe-Muster verwendet."
The blockquote cap is not here: `wikitool lint` reports it, so it is the contract's.
## Relationship labels
**Not this file's to list, and not localized.** A label is a machine value in `related:`, drawn
from [instructions/link-taxonomy.md](../instructions/link-taxonomy.md) and authorised per
destination in each `kb/<name>/COLLECTION.md`'s `outbound:` block. `- **depends-on:** [[Hermes]]`
is what a German page carries, and that is deliberate: the label is an identifier, so translating
it would make the graph's semantics depend on the prose again.
## Confidence rubric
`confidence_base` is set by hand and `confidence` is derived from it - that mechanism is the
contract's. What the number *means* is this instance's:
Base score for a single source is 0.5, adjusted by:
- **+0.2 per supporting source** (max +0.6)
- **+0.2** if confirmed <30 days ago, **+0.1** if <90 days
- **+0.1** for official documentation, **+0.05** for a reputable secondary source
- **+0.1** if multiple independent sources agree
In prose, hedge according to the score: below 0.6 write "möglicherweise"/"kann"; below 0.4
write "unsicher"/"unbestätigt".
## Keeping this file honest
Change it when a convention actually changes. `sections:` is safe to change at any time - the
regions are located by their markers and re-rendered under the new words on the next write.
`wikitool doctor` FAILs on a missing or unfilled file, and `wikitool docs verify` refuses a
`sections:` block that does not name both regions.
+89
View File
@@ -0,0 +1,89 @@
---
# wikitool:template-unfilled - delete this line once the file is answered.
language: en
profile: none
sections:
links: Relationships
footnotes: Footnotes
---
# kb/ - Authoring Conventions of This Instance
The decisions [kb/CONTRACT.md](CONTRACT.md) deliberately does not make. The contract holds what
the code enforces and is identical in every instance; this file holds what *this* instance
chooses, and no other instance has to agree with a word of it.
**It binds all the same.** Everything below applies to every page under `kb/`, exactly as the
contract does. The difference is ownership, not authority: a rule here is changed by editing
this file, a rule there by changing the stack.
Ready-made answers to every section below - including a complete German profile - are in
[instructions/kb-profiles.md](../instructions/kb-profiles.md). That catalogue is a palette, not
an enum: adopt an entry, adapt it, or write your own. What is written *here* is what holds.
The frontmatter above is the one machine-read part. `sections:` names the headings the two
generated regions render under. Safe to change at any time - each region is located by its
marker pair, so a rename re-renders words and nothing else.
## Language
Pages are written in **{language}**. This binds `kb/` and the authoring surface that shapes it -
the page type-specs `types/entity.md`, `types/concept.md`, `types/source.md` and
`types/comparison.md`, whose `## Template` blocks are the body skeleton every new page starts
from. `raw/` is untouched ([raw/CONTRACT.md](../raw/CONTRACT.md)), and the control plane stays
English: `AGENTS.md`, the stage contracts, this file, `instructions/`, and the type-specs for
non-page artifacts.
Which line is prose and which is an identifier - and therefore what is translated at all - is
the contract's rule, not this file's: see
[kb/CONTRACT.md § Language and identifiers](CONTRACT.md#language-and-identifiers).
{Which established foreign-language technical terms stay untranslated inside this instance's
prose, and where the vocabulary for that is looked up. Delete this paragraph if the KB language
is the one those terms are already in.}
### Section headings
The two generated regions render under the frontmatter's headings. An author never writes inside
them - they are rebuilt from frontmatter on every write. Any *other* heading is ordinary prose.
## Naming
- {Title form - words and spaces, or kebab-case, or the subject's own spelling.}
- {Singular or plural for entities.}
- {How a comparison page's title reads.}
- {The ADR prefix, if this instance files decisions as pages.}
- {What to name a thing: projects, systems, tools, technologies, people.}
The one naming fact that is *not* a choice, and therefore lives in the contract: the filename
stem is the page title, and `[[wikilinks]]` must match it exactly.
## Tone
{The register pages are written in, in one line.}
- {Words and constructions this instance refuses, with examples in the KB language.}
Good: {one sentence that is what this instance wants.}
Bad: {the same sentence written the way it must not be.}
## Relationship labels
**Not this file's to list, and not localized.** A label is a machine value in `related:`, drawn
from [instructions/link-taxonomy.md](../instructions/link-taxonomy.md) and authorised per
destination in each `kb/<name>/COLLECTION.md`'s `outbound:` block.
## Confidence rubric
`confidence_base` is set by hand and `confidence` is derived from it - that mechanism is the
contract's. What the number *means* is this instance's:
{the base score, what raises it, and by how much}
{How to hedge in prose at a low score, in the KB language.}
## Keeping this file honest
Change it when a convention actually changes. `wikitool doctor` FAILs on a missing or unfilled
file, and `wikitool docs verify` refuses a `sections:` block that does not name both regions.
+28 -7
View File
@@ -1,3 +1,10 @@
---
profile: comparisons
outbound:
any: [compares-with, contrasts, see-also]
required_by_stack: false
---
# kb/comparisons/ - Collection Contract # kb/comparisons/ - Collection Contract
Structured head-to-head evaluations of two or more things that already have pages here. A Structured head-to-head evaluations of two or more things that already have pages here. A
@@ -7,8 +14,10 @@ comparison exists so that neither subject's own page has to argue against the ot
That needs named, checkable dimensions and a stated trade-off; a page that lists differences That needs named, checkable dimensions and a stated trade-off; a page that lists differences
without saying what they cost has described, not compared. without saying what they cost has described, not compared.
Inherits [kb/CONTRACT.md](../CONTRACT.md) - naming, tone, linking, provenance and confidence Inherits [kb/CONTRACT.md](../CONTRACT.md) for the rules the stack enforces - linking mechanics,
are defined there and are not restated here. provenance, citation, the confidence machinery - and
[kb/CONVENTIONS.md](../CONVENTIONS.md) for what this instance decided: language, naming forms,
tone, relationship labels, the confidence rubric. Neither is restated here.
## Types offered ## Types offered
@@ -16,8 +25,9 @@ are defined there and are not restated here.
## Naming ## Naming
The title reads as a comparison: `Go vs Rust.md`, `Traefik vs nginx.md`. Order the subjects as The title form is [kb/CONVENTIONS.md § Naming](../CONVENTIONS.md#naming)'s. What is local here
they are most commonly spoken, not alphabetically. is the ordering: name the subjects as they are most commonly spoken together, not
alphabetically.
## Requirements ## Requirements
@@ -29,11 +39,22 @@ they are most commonly spoken, not alphabetically.
- State the trade-off, not a winner. Where a recommendation is genuinely warranted, scope it: - State the trade-off, not a winner. Where a recommendation is genuinely warranted, scope it:
"for X workload", not "better". "for X workload", not "better".
## Authorised labels
The `outbound:` block above is what `wikitool lint` and `xref add` check: which labels a page in
this collection may use, per destination. The catalogue they are drawn from - and what each one
asserts - is [instructions/link-taxonomy.md](../../instructions/link-taxonomy.md), which binds
nothing on its own.
Narrow for the opposite reason: a comparison's substance is its table, and its links to the compared subjects are the one relationship it asserts.
Adding a label here is a deliberate contract change, not a way around a refusal.
## Outbound linking ## Outbound linking
A comparison links to every subject with `related to`, and each subject links back. Comparison A comparison links to every subject with `compares-with`. The subjects do not have to link back:
pages are **exempt from the orphan check** - they are reached through `index.md` rather than a comparison is reached through the catalog, and each subject's inbound view renders the edge
through inbound prose links. anyway. Comparison pages are **exempt from the orphan check** for the same reason.
## What does not belong here ## What does not belong here
+29 -6
View File
@@ -1,3 +1,13 @@
---
profile: concepts
outbound:
concepts: [extends, grounds, rests-on, enables, precondition, exemplifies, abstracted-from, contrasts, compares-with, contradicts, composition, part-of, supersedes, derived-from, adapted-from, see-also]
entities: [operationalized-from, mechanism, procedure, applies-when, operates-on, invokes, exemplifies, see-also]
sources: [evidenced-by, derived-from, adapted-from, defined-in, see-also]
comparisons: [compares-with, see-also]
required_by_stack: false
---
# kb/concepts/ - Collection Contract # kb/concepts/ - Collection Contract
Ideas rather than things: architectures, patterns, protocols, workflows, recurring problems, Ideas rather than things: architectures, patterns, protocols, workflows, recurring problems,
@@ -8,8 +18,10 @@ records *what*.
without the reader having to open the entity pages that use it. If the explanation only makes without the reader having to open the entity pages that use it. If the explanation only makes
sense once you already know the system, it is on the wrong page. sense once you already know the system, it is on the wrong page.
Inherits [kb/CONTRACT.md](../CONTRACT.md) - naming, tone, linking, provenance and confidence Inherits [kb/CONTRACT.md](../CONTRACT.md) for the rules the stack enforces - linking mechanics,
are defined there and are not restated here. provenance, citation, the confidence machinery - and
[kb/CONVENTIONS.md](../CONVENTIONS.md) for what this instance decided: language, naming forms,
tone, relationship labels, the confidence rubric. Neither is restated here.
## Types offered ## Types offered
@@ -17,8 +29,8 @@ are defined there and are not restated here.
## Decisions and ADRs ## Decisions and ADRs
An architectural decision is a concept page prefixed `adr-NNN-`, e.g. An architectural decision is a concept page, prefixed as
`adr-001-use-go-modules.md`. It records: [kb/CONVENTIONS.md § Naming](../CONVENTIONS.md#naming) says. It records:
- **Context** - what forced a decision. - **Context** - what forced a decision.
- **Decision** - what was chosen. - **Decision** - what was chosen.
@@ -26,8 +38,19 @@ An architectural decision is a concept page prefixed `adr-NNN-`, e.g.
- **Status** - proposed / accepted / deprecated / superseded. - **Status** - proposed / accepted / deprecated / superseded.
- Links to every entity the decision affects. - Links to every entity the decision affects.
A superseded ADR is never deleted or rewritten; a new one supersedes it and both link to the A superseded ADR is never deleted or rewritten. The new one declares `supersedes` pointing at
other with `replaces` / `replaced by`. it; the old one needs no edge back, because its inbound view renders the replacement.
## Authorised labels
The `outbound:` block above is what `wikitool lint` and `xref add` check: which labels a page in
this collection may use, per destination. The catalogue they are drawn from - and what each one
asserts - is [instructions/link-taxonomy.md](../../instructions/link-taxonomy.md), which binds
nothing on its own.
The widest authorisation in this instance, because argumentation is what concept pages do. Note that the operational labels are absent: a concept does not `depend-on` anything - the entity implementing it does.
Adding a label here is a deliberate contract change, not a way around a refusal.
## Outbound linking ## Outbound linking
@@ -0,0 +1,73 @@
---
type: types/concept.md
concept_type: decision
tags: []
created: 2026-09-01
modified: 2026-09-01
related: [Chemenu]
sources: ['Source - Public Release, Corpus Purge and History Squash Session 2026-09-01', Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01]
confidence: 0.70
confidence_base: 0.70
provenance: sourced
summary: 'Private Korpusinhalte per Loeschung entfernen statt zu anonymisieren: ein Seitentitel ist der einzige Identifier eines Wikis, Umbenennen ist die volle page-lifecycle-Prozedur je Seite, Loeschen ist ein unterstuetztes Kommando.'
---
# Delete Rather Than Anonymize
**Typ:** Decision
## Definition
Wenn private oder sensible Inhalte aus einem Wiki entfernt werden müssen, ist Löschen einer
zugehörigen Seite in der Regel dem Anonymisieren (Umbenennen, Ersetzen sensibler Details bei
sonst unverändertem Inhalt) vorzuziehen - wenn ein unterstütztes Löschkommando existiert.
## Kernpunkte
- Ein Seitentitel ist in einem verlinkten Wiki oft der **einzige Identifier** einer Seite: Er
lebt in Wikilinks, Zitatmarkern und Frontmatter-Arrays jeder referenzierenden Seite. Ihn zu
ändern (Anonymisieren durch Umbenennen) verlangt deshalb eine vollständige Rename-Prozedur pro
betroffener Seite - bei mehreren zusammenhängenden Seiten multipliziert sich der Aufwand.
- Löschen dagegen ist ein einzelner, unterstützter Vorgang, der eine Seite mechanisch aus dem
Rest des Wikis de-linkt (bekannte Referenzarten: Frontmatter-Felder, ganzzeilige
Verweis-Aufzählungen). Er ist damit für strukturelle Bereinigung **schneller und weniger
fehleranfällig** als Anonymisierung.
- Bei Inhalten, die eine reale Topologie beschreiben (z. B. eine Infrastrukturdokumentation),
entschärft Anonymisieren einzelner Bezeichner (Hostnamen, IP-Adressen) die eigentliche
Preisgabe nicht: Die Struktur - welche Systeme wie zusammenhängen - bleibt erhalten, auch wenn
die Namen ausgetauscht sind.
- **Grenze der Methode:** Ein mechanisches Löschkommando entfernt typischerweise nur
strukturelle Referenzen (Frontmatter, Aufzählungen), nicht zwingend Erwähnungen im Fließtext
einer anderen Seite. Nach der Löschung ist eine gezielte Nachkontrolle nötig, ob der entfernte
Name noch im Klartext irgendwo im Wiki steht.
## Wann zu verwenden
- Der zu entfernende Inhalt ist als eigenständige Seite oder eigenständige Seitengruppe
abgrenzbar.
- Ein Löschkommando existiert, das Referenzen mechanisch bereinigt (nicht ein bloßes Entfernen
der Datei, das tote Links hinterlässt).
- Der Inhalt beschreibt eine reale, zusammenhängende Struktur (Infrastruktur, ein Netzwerk, eine
Organisation), bei der einzelne Bezeichner austauschen die eigentliche Preisgabe nicht behebt.
## Wann NICHT zu verwenden
- Wenn nur ein einzelner sensibler Wert innerhalb einer sonst wertvollen, generischen Seite
steht (z. B. ein Firmenname als Beispiel in einer sonst allgemeingültigen Anleitung) - dort ist
gezieltes Redigieren der Seite treffender als sie komplett zu verwerfen.
- Wenn die Seite Beziehungen trägt, die für sich genommen wertvoll und nicht sensibel sind - dann
kann eine Neufassung mit generischem Beispiel sinnvoller sein als Löschung.
## Verwandte Concepts
- [[Mass-Update Gate]]
## Beziehungen
- **gilt fuer:** [[Chemenu]]
## Siehe auch
- [[Source - Public Release, Corpus Purge and History Squash Session 2026-09-01]]
- [[Chemenu]]
- [[Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01]]
@@ -0,0 +1,71 @@
---
type: types/concept.md
concept_type: decision
tags: []
created: 2026-09-01
modified: 2026-09-01
related: [Chemenu]
sources: ['Source - Public Release, Corpus Purge and History Squash Session 2026-09-01']
confidence: 0.70
confidence_base: 0.70
provenance: sourced
summary: Ein Repo mit Code- und Inhaltsanteil erhaelt zwei Lizenzen; die Grenze zwischen ihnen ist kein zweiter, gepflegter Pfadkatalog, sondern der ohnehin vorhandene Dateiplan des Distributionswerkzeugs.
---
# Dual Licensing by File Plan
**Typ:** Decision
## Definition
Ein Repository, das sowohl Werkzeug-Code als auch inhaltliches Material (Dokumentation, Daten,
kompiliertes Wissen) enthält, bekommt zwei Lizenzdateien statt einer - eine für den Code, eine
für den Inhalt. Welche Datei zu welcher Lizenz gehört, wird nicht in einer eigenen, zweiten
Liste festgehalten, sondern aus dem Dateiplan abgeleitet, den ein vorhandenes
Distributions-/Build-Werkzeug ohnehin pflegt.
## Kernpunkte
- Der naheliegende Fehler ist, die Grenze zwischen „Code" und „Inhalt" als eigene, gepflegte
Aufzählung von Pfaden in der Lizenzdatei selbst festzuschreiben. Das ist eine zweite Kopie
einer Regel, die bereits an anderer Stelle existiert (dem Dateiplan des Build-/
Distributionswerkzeugs) - und die Kopie, die driftet, wenn sich Verzeichnisse verschieben.
- Stattdessen verweist die Lizenz-Notiz auf den bestehenden Plan (z. B. eine Funktion, die
berechnet, was in eine Distribution exportiert wird und was nicht) als **einzige** Quelle der
Wahrheit für die Grenze.
- Welche der beiden Lizenzen den generischen Dateinamen `LICENSE` trägt, ist keine
Nebensächlichkeit: Es sollte die Lizenz sein, die ein Code-Hosting-Dienst (Forge) für das
Repository insgesamt meldet - typischerweise die restriktivere/Copyleft-Lizenz. Ein Leser, der
eine Copyleft-Pflicht übersieht, wird dadurch geschädigt; wer eine Pflicht zu viel annimmt,
nicht.
- Ein Distributions-Export, der Code unter einer Copyleft-Lizenz ausliefert, muss die
zugehörige Lizenzdatei zwingend mitliefern (nicht optional, nicht still übersprungen, wenn sie
fehlt) - sonst ist die exportierte Instanz eine Lizenzverletzung, sobald sie veröffentlicht
wird.
## Wann zu verwenden
- Ein Repository trägt sowohl Software-/Werkzeugcode als auch Inhalt mit eigenem
Urheberrechtscharakter (Dokumentation, Wissensbasis, Daten), für die unterschiedliche Lizenzen
angemessen sind.
- Es existiert bereits ein Werkzeug, das programmatisch entscheidet, welche Dateien zu welcher
Kategorie gehören (z. B. für einen Export- oder Build-Schritt).
## Wann NICHT zu verwenden
- Bei einem Repository, dessen Inhalt untrennbar mit dem Code verwoben ist und für das keine
separate, maschinell nachvollziehbare Grenze existiert - dort wäre die Lizenz-Zuordnung selbst
wieder eine unabhängige, drift-anfällige Liste.
## Verwandte Concepts
- [[Chemenu]]
## Beziehungen
- **gilt fuer:** [[Chemenu]]
## Siehe auch
- [[Source - Public Release, Corpus Purge and History Squash Session 2026-09-01]]
- [[Chemenu]]
+7 -3
View File
@@ -2,7 +2,7 @@
# kb/concepts/ - Index # kb/concepts/ - Index
76 page(s). Regenerated by `wikitool index rebuild`. 80 page(s). Regenerated by `wikitool index rebuild`.
## All ## All
@@ -25,9 +25,11 @@
| [[CPPC]] | protocol | Hardwareschnittstelle Collaborative Processor Performance Control für feingranulares CPU-Power-Management zwischen Betriebssystem und AMD-Prozessor. | 2026-08-29 | | [[CPPC]] | protocol | Hardwareschnittstelle Collaborative Processor Performance Control für feingranulares CPU-Power-Management zwischen Betriebssystem und AMD-Prozessor. | 2026-08-29 |
| [[Cross-platform Agent Skills]] | architecture | Architektur fuer Agent-Skills, die ueber mehrere LLM-Werkzeuge hinweg funktionieren; in Chemenu selbst am 2026-08-04 umgesetzt und ueberprueft | 2026-09-01 | | [[Cross-platform Agent Skills]] | architecture | Architektur fuer Agent-Skills, die ueber mehrere LLM-Werkzeuge hinweg funktionieren; in Chemenu selbst am 2026-08-04 umgesetzt und ueberprueft | 2026-09-01 |
| [[Crystallization]] | workflow | Verdichten abgeschlossener Erkundungen, Debugging-Sitzungen und Recherchen zu strukturierten Wiki-Auszügen als eigenständige Wissensquellen. | 2026-08-29 | | [[Crystallization]] | workflow | Verdichten abgeschlossener Erkundungen, Debugging-Sitzungen und Recherchen zu strukturierten Wiki-Auszügen als eigenständige Wissensquellen. | 2026-08-29 |
| [[Delete Rather Than Anonymize]] | decision | Private Korpusinhalte per Loeschung entfernen statt zu anonymisieren: ein Seitentitel ist der einzige Identifier eines Wikis, Umbenennen ist die volle page-lifecycle-Prozedur je Seite, Loeschen ist ein unterstuetztes Kommando. | 2026-09-01 |
| [[Denylist over Allowlist]] | decision | Entscheidung, schreibbare Felder als Schema minus kurzer Sperrliste zu bestimmen statt als gepflegte Positivliste, weil die Positivliste eine zweite Kopie des Schemas waere | 2026-08-31 | | [[Denylist over Allowlist]] | decision | Entscheidung, schreibbare Felder als Schema minus kurzer Sperrliste zu bestimmen statt als gepflegte Positivliste, weil die Positivliste eine zweite Kopie des Schemas waere | 2026-08-31 |
| [[Detect-Repair Asymmetry]] | problem | Werkzeugluecke, in der ein Check einen Defekt zuverlaessig meldet, aber kein Befehl ihn behebt - womit die Handeditierung der einzige verbleibende Ausweg ist | 2026-08-31 | | [[Detect-Repair Asymmetry]] | problem | Werkzeugluecke, in der ein Check einen Defekt zuverlaessig meldet, aber kein Befehl ihn behebt - womit die Handeditierung der einzige verbleibende Ausweg ist | 2026-08-31 |
| [[Diff-Reviewable Agent Edits]] | decision | Entscheidung, Dateiaenderungen ueber Edit/Write statt ueber Shell-Heredocs zu fahren, weil nur das erste eine pruefbare Diff hinterlaesst | 2026-08-31 | | [[Diff-Reviewable Agent Edits]] | decision | Entscheidung, Dateiaenderungen ueber Edit/Write statt ueber Shell-Heredocs zu fahren, weil nur das erste eine pruefbare Diff hinterlaesst | 2026-08-31 |
| [[Dual Licensing by File Plan]] | decision | Ein Repo mit Code- und Inhaltsanteil erhaelt zwei Lizenzen; die Grenze zwischen ihnen ist kein zweiter, gepflegter Pfadkatalog, sondern der ohnehin vorhandene Dateiplan des Distributionswerkzeugs. | 2026-09-01 |
| [[Entity Extraction]] | pattern | Erkennen und Strukturieren von Entities (Personen, Projekte, Bibliotheken, Concepts, Dateien, Entscheidungen, Systeme, Werkzeuge) samt typspezifischer Attribute aus Rohquellen. | 2026-08-29 | | [[Entity Extraction]] | pattern | Erkennen und Strukturieren von Entities (Personen, Projekte, Bibliotheken, Concepts, Dateien, Entscheidungen, Systeme, Werkzeuge) samt typspezifischer Attribute aus Rohquellen. | 2026-08-29 |
| [[Episodic Memory]] | architecture | Speicherschicht für verdichtete Sitzungszusammenfassungen und Befunde; Brücke zwischen rohem Working Memory und langlebigem Semantic Memory. | 2026-08-29 | | [[Episodic Memory]] | architecture | Speicherschicht für verdichtete Sitzungszusammenfassungen und Befunde; Brücke zwischen rohem Working Memory und langlebigem Semantic Memory. | 2026-08-29 |
| [[Event-Driven Automation]] | workflow | Muster, das automatische Auslöser an Wiki-Lebenszyklusereignisse hängt, um manuellen Pflegeaufwand und das Risiko der Verwahrlosung zu senken. | 2026-08-29 | | [[Event-Driven Automation]] | workflow | Muster, das automatische Auslöser an Wiki-Lebenszyklusereignisse hängt, um manuellen Pflegeaufwand und das Risiko der Verwahrlosung zu senken. | 2026-08-29 |
@@ -40,14 +42,15 @@
| [[Implementation Spectrum]] | architecture | Modularer Einführungspfad für die Funktionen von LLM Wiki v2, vom minimal tragfähigen Wiki bis zur vollen Umsetzung mit Automatisierung und Governance. | 2026-08-29 | | [[Implementation Spectrum]] | architecture | Modularer Einführungspfad für die Funktionen von LLM Wiki v2, vom minimal tragfähigen Wiki bis zur vollen Umsetzung mit Automatisierung und Governance. | 2026-08-29 |
| [[Index Scaling]] | workflow | Skalierungsregeln für Indexseiten: Tabellenabschnitte ab 50 Einträgen teilen, ab 200 Seiten _meta/topic-map.md anlegen | 2026-08-29 | | [[Index Scaling]] | workflow | Skalierungsregeln für Indexseiten: Tabellenabschnitte ab 50 Einträgen teilen, ab 200 Seiten _meta/topic-map.md anlegen | 2026-08-29 |
| [[Issue Label Scheme]] | decision | Zweiachsiges Pflicht-Labelschema fuer das Gitea-Board: prio/1..3 und size/XS..L, bewusst keine dritte Achse; die Regel liegt in instructions/dev/, weil sie keine ausgelieferte Instanz erreichen darf | 2026-08-31 | | [[Issue Label Scheme]] | decision | Zweiachsiges Pflicht-Labelschema fuer das Gitea-Board: prio/1..3 und size/XS..L, bewusst keine dritte Achse; die Regel liegt in instructions/dev/, weil sie keine ausgelieferte Instanz erreichen darf | 2026-08-31 |
| [[Iteration and Cost Limits]] | workflow | Im Code durchgesetzte Obergrenze von 60 wikitool-Aufrufen je Session\, Loop-Breaker bei 3 identischen Wiederholungen\, Slot-Erstattung bei abgelehntem Aufruf und ein seit 1.5.0 gemessenes Kalibrierungsband von 5-15 (einfach) bzw. 20-35 (komplex) Aufrufen | 2026-08-31 | | [[Iteration and Cost Limits]] | workflow | Im Code durchgesetzte Obergrenze von 60 wikitool-Aufrufen je Session, Loop-Breaker bei 3 identischen Wiederholungen, Slot-Erstattung, ein gemessenes Kalibrierungsband, und Retrieval sowie der MCP-Leseserver bleiben ausgenommen | 2026-09-02 |
| [[KB Migration]] | workflow | Migration des KB-Inhalts entlang einer geordneten Versionskette; abgegrenzt gegen offene Instanz-Aktionen, die in den doctor-Check gehoeren statt in die Kette | 2026-08-31 | | [[KB Migration]] | workflow | Migration des KB-Inhalts entlang einer geordneten Versionskette; abgegrenzt gegen offene Instanz-Aktionen, die in den doctor-Check gehoeren statt in die Kette | 2026-08-31 |
| [[KB Stack Versioning]] | decision | Semantische Versionierung des Wiki-Stacks: VERSION beschreibt die Maschinerie, Kompatibilitaet ist die linkeste Nicht-Null-Komponente, und drei getrennte Dateien trennen Maschinerie, Herkunft und Content-Form | 2026-08-30 | | [[KB Stack Versioning]] | decision | Semantische Versionierung des Wiki-Stacks: VERSION beschreibt die Maschinerie, Kompatibilitaet (Drop-in-Ersatz) und Inhaltsmigration sind seit 2.5.0 getrennte, unabhaengig geprueft Fragen | 2026-09-02 |
| [[Knowledge Compounding]] | workflow | Effekt, bei dem Wissen im Wiki an Wert gewinnt, weil jede neue Quelle an bestehende, untereinander verwiesene Seiten anknüpft und sie ergänzt. | 2026-08-29 | | [[Knowledge Compounding]] | workflow | Effekt, bei dem Wissen im Wiki an Wert gewinnt, weil jede neue Quelle an bestehende, untereinander verwiesene Seiten anknüpft und sie ergänzt. | 2026-08-29 |
| [[Knowledge Graph]] | architecture | Typisierte Schicht aus Entities und Beziehungen über den Wiki-Seiten, die eine reichere Wissensdarstellung und graphbasierte Abfragen ermöglicht. | 2026-08-29 | | [[Knowledge Graph]] | architecture | Typisierte Schicht aus Entities und Beziehungen über den Wiki-Seiten, die eine reichere Wissensdarstellung und graphbasierte Abfragen ermöglicht. | 2026-08-29 |
| [[Lint Workflow]] | workflow | Deterministischer Health-Check rund um wikitool lint; seit 1.7.2 maskiert es Code vor dem Notation-Match und zaehlt Zitat-Bloecke statt Zeilen | 2026-09-01 | | [[Lint Workflow]] | workflow | Deterministischer Health-Check rund um wikitool lint; seit 1.7.2 maskiert es Code vor dem Notation-Match und zaehlt Zitat-Bloecke statt Zeilen | 2026-09-01 |
| [[LLM Wiki Pattern]] | architecture | Methodik für persönliches Wissensmanagement, bei der ein LLM aus Rohquellen ein dauerhaftes Wiki aufbaut - Wissen wird kompiliert statt per RAG neu hergeleitet. | 2026-08-29 | | [[LLM Wiki Pattern]] | architecture | Methodik für persönliches Wissensmanagement, bei der ein LLM aus Rohquellen ein dauerhaftes Wiki aufbaut - Wissen wird kompiliert statt per RAG neu hergeleitet. | 2026-08-29 |
| [[Mass-Update Gate]] | workflow | Mass-Update Gate: publish endet mit 42 (Freigabe durch den Menschen noetig) ab 10 gezaehlten Dateien; generierte Dateien und work/ werden committet\, aber seit 1.5.0 nicht gezaehlt; freigegeben per --confirm <token> | 2026-09-01 | | [[Mass-Update Gate]] | workflow | Mass-Update Gate: publish endet mit 42 (Freigabe durch den Menschen noetig) ab 10 gezaehlten Dateien; generierte Dateien und work/ werden committet\, aber seit 1.5.0 nicht gezaehlt; freigegeben per --confirm <token> | 2026-09-01 |
| [[MCP-Leseserver]] | architecture | Zweiter Konsument von chemenu ueber MCP: search/types/describe_type/lint/status auf chemenu.api.Corpus, strukturell ohne Schreibpfad, jede Antwort trage einen Commit-Stempel. | 2026-09-02 |
| [[Memory Lifecycle]] | architecture | Architektur des Wissenslebenszyklus mit Confidence Scoring, Supersession, Forgetting und Consolidation Tiers zur Pflege von Fakten über die Zeit. | 2026-08-29 | | [[Memory Lifecycle]] | architecture | Architektur des Wissenslebenszyklus mit Confidence Scoring, Supersession, Forgetting und Consolidation Tiers zur Pflege von Fakten über die Zeit. | 2026-08-29 |
| [[Mesh Sync]] | pattern | Abgleichsmechanismus, der Beobachtungen paralleler Agenten in ein gemeinsames Wiki überführt; Last-Write-Wins mit Konflikterkennung und manuellem Eingriff. | 2026-08-29 | | [[Mesh Sync]] | pattern | Abgleichsmechanismus, der Beobachtungen paralleler Agenten in ein gemeinsames Wiki überführt; Last-Write-Wins mit Konflikterkennung und manuellem Eingriff. | 2026-08-29 |
| [[Modbus]] | protocol | Industrielles Kommunikationsprotokoll von 1979 zur Anbindung speicherprogrammierbarer Steuerungen und Geräte über serielle oder TCP-Netze. | 2026-08-29 | | [[Modbus]] | protocol | Industrielles Kommunikationsprotokoll von 1979 zur Anbindung speicherprogrammierbarer Steuerungen und Geräte über serielle oder TCP-Netze. | 2026-08-29 |
@@ -58,6 +61,7 @@
| [[Personalization Plane]] | architecture | Schicht fuer Instanz-Identitaet: USER.md/SOUL.md werden als Template ausgeliefert, im Setup-Interview woertlich befuellt und vom doctor-Check auf fehlend wie unbefuellt geprueft | 2026-08-31 | | [[Personalization Plane]] | architecture | Schicht fuer Instanz-Identitaet: USER.md/SOUL.md werden als Template ausgeliefert, im Setup-Interview woertlich befuellt und vom doctor-Check auf fehlend wie unbefuellt geprueft | 2026-08-31 |
| [[Privacy and Governance]] | workflow | Rahmenwerk zur Absicherung von Wiki-Inhalten über Datenfilterung beim Ingest, Audit-Trail-Protokollierung und umkehrbare Massenoperationen. | 2026-08-29 | | [[Privacy and Governance]] | workflow | Rahmenwerk zur Absicherung von Wiki-Inhalten über Datenfilterung beim Ingest, Audit-Trail-Protokollierung und umkehrbare Massenoperationen. | 2026-08-29 |
| [[Procedural Memory]] | architecture | Langlebigste Speicherschicht für Abläufe, Muster, bewährte Vorgehensweisen und Rezepte, gewonnen aus wiederholten semantischen Beobachtungen. | 2026-08-29 | | [[Procedural Memory]] | architecture | Langlebigste Speicherschicht für Abläufe, Muster, bewährte Vorgehensweisen und Rezepte, gewonnen aus wiederholten semantischen Beobachtungen. | 2026-08-29 |
| [[Publish-Remote Gate]] | workflow | Drittes, im Code durchgesetztes Gate: publish bricht mit Exit 42 ab, wenn die aufgeloeste Push-URL nicht in einer optionalen, gitignoreten Allowlist steht; doctor benennt seit 2026-09-02 den Gate-Zustand statt nur die Dateiexistenz | 2026-09-02 |
| [[Quality and Self-Correction]] | workflow | Automatische Qualitätssicherung für Wikis mit Inhaltsbewertung, Selbstheilung und Widerspruchserkennung. | 2026-08-29 | | [[Quality and Self-Correction]] | workflow | Automatische Qualitätssicherung für Wikis mit Inhaltsbewertung, Selbstheilung und Widerspruchserkennung. | 2026-08-29 |
| [[Quality Scoring]] | pattern | Quantitative Bewertung aller vom LLM geschriebenen Inhalte nach struktureller Qualität, Vollständigkeit der Quellenangaben, Konsistenz mit dem Wiki und Themenabdeckung. | 2026-08-29 | | [[Quality Scoring]] | pattern | Quantitative Bewertung aller vom LLM geschriebenen Inhalte nach struktureller Qualität, Vollständigkeit der Quellenangaben, Konsistenz mit dem Wiki und Themenabdeckung. | 2026-08-29 |
| [[RAG]] | architecture | Architekturmuster, bei dem LLMs die Generierung um Dokumente aus einer Wissensbasis anreichern. | 2026-08-29 | | [[RAG]] | architecture | Architekturmuster, bei dem LLMs die Generierung um Dokumente aus einer Wissensbasis anreichern. | 2026-08-29 |
+12 -4
View File
@@ -3,13 +3,13 @@ type: types/concept.md
concept_type: workflow concept_type: workflow
tags: [gate, safety, iteration-budget, loop-breaker] tags: [gate, safety, iteration-budget, loop-breaker]
created: 2026-08-07 created: 2026-08-07
modified: 2026-08-31 modified: 2026-09-02
related: [Mass-Update Gate, Anti-Cramming Heuristic, Index Scaling, wikitool, Structural Enforcement over Documented Rule] related: [Mass-Update Gate, Anti-Cramming Heuristic, Index Scaling, wikitool, Structural Enforcement over Documented Rule, MCP-Leseserver]
sources: [Source - LLM Improvements Production Agent Gaps 2026, Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31, Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31] sources: [Source - LLM Improvements Production Agent Gaps 2026, Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31, Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31, Source - MCP Read Server Implementation Session 2026-09-02]
confidence: 0.88 confidence: 0.88
confidence_base: 0.88 confidence_base: 0.88
provenance: sourced provenance: sourced
summary: Im Code durchgesetzte Obergrenze von 60 wikitool-Aufrufen je Session\, Loop-Breaker bei 3 identischen Wiederholungen\, Slot-Erstattung bei abgelehntem Aufruf und ein seit 1.5.0 gemessenes Kalibrierungsband von 5-15 (einfach) bzw. 20-35 (komplex) Aufrufen summary: Im Code durchgesetzte Obergrenze von 60 wikitool-Aufrufen je Session, Loop-Breaker bei 3 identischen Wiederholungen, Slot-Erstattung, ein gemessenes Kalibrierungsband, und Retrieval sowie der MCP-Leseserver bleiben ausgenommen
--- ---
# Iteration and Cost Limits # Iteration and Cost Limits
@@ -48,6 +48,11 @@ Eine hart in Code durchgesetzte Obergrenze für die Anzahl der Tool-Aufrufe, die
- Einzelne, begrenzte Einmalvorgänge, bei denen die Aufrufen-Anzahl inhärent festgelegt ist (z. B. ein einzelner `new entity`-Aufruf) - das Gate wird dort immer noch gleichmäßig angewendet, wird aber im Wesentlichen nie ausgelöst. - Einzelne, begrenzte Einmalvorgänge, bei denen die Aufrufen-Anzahl inhärent festgelegt ist (z. B. ein einzelner `new entity`-Aufruf) - das Gate wird dort immer noch gleichmäßig angewendet, wird aber im Wesentlichen nie ausgelöst.
- Als Ersatz für das [[Mass-Update Gate]], das auf den *Schadensradius* eines `publish` (Dateien, die von einem einzelnen Push betroffen sind) begrenzt ist, nicht auf die *Iterationsmenge* über eine Sitzung - die beiden Gates beheben unterschiedliche Ausfallmodi und beide bleiben notwendig. - Als Ersatz für das [[Mass-Update Gate]], das auf den *Schadensradius* eines `publish` (Dateien, die von einem einzelnen Push betroffen sind) begrenzt ist, nicht auf die *Iterationsmenge* über eine Sitzung - die beiden Gates beheben unterschiedliche Ausfallmodi und beide bleiben notwendig.
- Im [[MCP-Leseserver]]. Das Gate begrenzt eine Agenten-Session am unbemerkten Iterieren über den
Wiki-Zustand - deshalb ist Retrieval bereits generell ausgenommen (`SKIP_COMMANDS`) -, nicht
einen Nutzer, der oft sucht. Ein zu häufig suchender Nutzer ist ein Ressourcenproblem, das vor
den Serverprozess gehört (Rate Limiting), nicht in dieses Gate - beide zu vermischen würde es
zu einem Rate Limiter verwässern.[^s-mcp-read-server-implementation-session-2026-09-02]
## Verwandte Concepts ## Verwandte Concepts
@@ -63,6 +68,7 @@ Eine hart in Code durchgesetzte Obergrenze für die Anzahl der Tool-Aufrufe, die
- **schützt:** [[Index Scaling]] - **schützt:** [[Index Scaling]]
- **implementiert durch:** [[wikitool]] - **implementiert durch:** [[wikitool]]
- **wendet an:** [[Structural Enforcement over Documented Rule]] - **wendet an:** [[Structural Enforcement over Documented Rule]]
- **nimmt aus:** [[MCP-Leseserver]]
## Siehe auch ## Siehe auch
@@ -73,9 +79,11 @@ Eine hart in Code durchgesetzte Obergrenze für die Anzahl der Tool-Aufrufe, die
- [[Source - LLM Improvements Production Agent Gaps 2026]] - [[Source - LLM Improvements Production Agent Gaps 2026]]
- [[Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31]] - [[Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31]]
- [[Structural Enforcement over Documented Rule]] - [[Structural Enforcement over Documented Rule]]
- [[MCP-Leseserver]]
## Fußnoten ## Fußnoten
[^s-llm-improvements-production-agent-gaps-2026]: [[Source - LLM Improvements Production Agent Gaps 2026]] [^s-llm-improvements-production-agent-gaps-2026]: [[Source - LLM Improvements Production Agent Gaps 2026]]
[^s-conversation-gate-counting-and-measured-calibration-session-2026-08-31]: [[Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31]] [^s-conversation-gate-counting-and-measured-calibration-session-2026-08-31]: [[Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31]]
[^s-conversation-comma-bug-budget-refund-and-lint-report-path-session-2026-08-31]: [[Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31]] [^s-conversation-comma-bug-budget-refund-and-lint-report-path-session-2026-08-31]: [[Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31]]
[^s-mcp-read-server-implementation-session-2026-09-02]: [[Source - MCP Read Server Implementation Session 2026-09-02]]
+21 -6
View File
@@ -3,13 +3,13 @@ type: types/concept.md
concept_type: decision concept_type: decision
tags: [versioning, semver, release, stack] tags: [versioning, semver, release, stack]
created: 2026-08-30 created: 2026-08-30
modified: 2026-08-30 modified: 2026-09-02
related: [wikitool, Issue Label Scheme] related: [wikitool, Issue Label Scheme]
sources: [Source - Conversation - Versioning CI-CD and Content Migration Session 2026-08-30, Source - Conversation - Issue Triage Labels and TODO Retirement Session 2026-08-31] sources: [Source - Conversation - Versioning CI-CD and Content Migration Session 2026-08-30, Source - Conversation - Issue Triage Labels and TODO Retirement Session 2026-08-31, Source - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02]
confidence: 0.70 confidence: 0.70
confidence_base: 0.70 confidence_base: 0.70
provenance: sourced provenance: sourced
summary: 'Semantische Versionierung des Wiki-Stacks: VERSION beschreibt die Maschinerie, Kompatibilitaet ist die linkeste Nicht-Null-Komponente, und drei getrennte Dateien trennen Maschinerie, Herkunft und Content-Form' summary: 'Semantische Versionierung des Wiki-Stacks: VERSION beschreibt die Maschinerie, Kompatibilitaet (Drop-in-Ersatz) und Inhaltsmigration sind seit 2.5.0 getrennte, unabhaengig geprueft Fragen'
--- ---
# KB Stack Versioning # KB Stack Versioning
@@ -42,9 +42,18 @@ deshalb eine ausdrückliche Handlung.
Caret-Ranges Caret-Ranges
verwenden[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30]. Sie gilt verwenden[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30]. Sie gilt
einheitlich für `0.x` und `1.x`, sodass unter `0.x` der Schritt `0.1.x` -> `0.2.0` dasselbe einheitlich für `0.x` und `1.x`, sodass unter `0.x` der Schritt `0.1.x` -> `0.2.0` dasselbe
Migrationssignal trägt wie `MAJOR` ab `1.0.0`. Der Code für den `compat_key` ist deshalb Signal trägt wie `MAJOR` ab `1.0.0`. Der Code für den `compat_key` ist deshalb einheitlich
einheitlich formuliert und musste beim Wechsel auf `1.0.0` nicht angefasst formuliert und musste beim Wechsel auf `1.0.0` nicht angefasst
werden[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30]. werden[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30].
- **Kompatibilität und Inhaltsmigration sind zwei unabhängige Fragen, seit `2.5.0` auch zwei
getrennte Marker.** Kompatibilität fragt, ob die neue Version ein Drop-in-Ersatz ist -
vorwärts ohne Handarbeit, rückwärts noch downgradebar; Inhaltsmigration fragt, ob `kb/` sich
bewegen muss. Ein Grenzübertritt kann `kb/` unangetastet lassen und trotzdem MAJOR sein - der
`2.0.0`-Rebranding-Bump ist das Beispiel: Update-Pfad, Release-Artefaktname und
Paket-Import-Name brachen, keine Seite tat es. `version bump` verlangt deshalb bei jedem
Grenzübertritt `--breaking "<was aufhört zu funktionieren>"`, unabhängig von
`--no-migration`/einem Migrationsdokument; beide Zeilen landen getrennt im
`CHANGES.md`-Eintrag[^s-version-part-nomenclature-and-breaking-change-gate-session-2026-09-02].
- **`x.y.z` ist die maximale Granularität. Keine Pre-Release-Suffixe.** Eine zweite - **`x.y.z` ist die maximale Granularität. Keine Pre-Release-Suffixe.** Eine zweite
Ordnungsregel müsste vom Release-Feed, von der Migrationskette und von der Ordnungsregel müsste vom Release-Feed, von der Migrationskette und von der
Kompatibilitätsprüfung gleichermaßen befolgt Kompatibilitätsprüfung gleichermaßen befolgt
@@ -64,7 +73,11 @@ deshalb eine ausdrückliche Handlung.
Instanz[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30]. Instanz[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30].
- **Die Grenze wird an zwei Stellen erzwungen:** in `version bump` und in `docs verify`, ergänzt - **Die Grenze wird an zwei Stellen erzwungen:** in `version bump` und in `docs verify`, ergänzt
um einen `kb-version`-Check in um einen `kb-version`-Check in
`doctor`[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30]. `doctor`[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30]. Seit
`2.5.0` prüft `docs verify` dort zwei unabhängige Dinge - `check_migration_for_boundary` (hat
der Korpus sich bewegt) und `check_breaking_change_for_boundary` (wurde der Bruch benannt) -,
weil ein Grenzübertritt die eine Prüfung bestehen und an der anderen scheitern
kann[^s-version-part-nomenclature-and-breaking-change-gate-session-2026-09-02].
## Beispiele ## Beispiele
@@ -105,7 +118,9 @@ kann.
- [[wikitool]] - [[wikitool]]
- [[Issue Label Scheme]] - [[Issue Label Scheme]]
- [[Source - Conversation - Issue Triage Labels and TODO Retirement Session 2026-08-31]] - [[Source - Conversation - Issue Triage Labels and TODO Retirement Session 2026-08-31]]
- [[Source - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02]]
## Fußnoten ## Fußnoten
[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30]: [[Source - Conversation - Versioning CI-CD and Content Migration Session 2026-08-30]] [^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30]: [[Source - Conversation - Versioning CI-CD and Content Migration Session 2026-08-30]]
[^s-version-part-nomenclature-and-breaking-change-gate-session-2026-09-02]: [[Source - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02]]
+108
View File
@@ -0,0 +1,108 @@
---
type: types/concept.md
concept_type: architecture
tags: [mcp, library-boundary, search, server]
created: 2026-09-02
modified: 2026-09-02
related: [wikitool, Publish-Remote Gate, Mass-Update Gate, Iteration and Cost Limits, Chemenu]
sources: [Source - MCP Read Server Implementation Session 2026-09-02]
confidence: 0.50
confidence_base: 0.50
provenance: sourced
summary: 'Zweiter Konsument von chemenu ueber MCP: search/types/describe_type/lint/status auf chemenu.api.Corpus, strukturell ohne Schreibpfad, jede Antwort trage einen Commit-Stempel.'
---
# MCP-Leseserver
**Typ:** Architecture
## Definition
Ein zweiter Konsument desselben Kerns, nicht ein zweites Programm: `tools/chemenu/mcp/` exponiert
`search`, `types`, `describe_type`, `lint` und `status` über MCP, indem es dieselben Funktionen
aufruft, die `wikitool` auch aufruft - vermittelt durch `chemenu.api.Corpus`, den In-Process-
Einstiegspunkt. Ein Golden-Test hält die Ausgaben beider Wege gegeneinander, statt darauf zu
vertrauen, dass sie übereinstimmen.
## Kernpunkte
- **Kein Schreibpfad, strukturell.** Weder der Server noch `chemenu.api` importiert etwas unter
`chemenu.commands` - `new`, `touch`, `xref`, `publish`, `migrate` sind aus diesem Prozess
heraus nicht erreichbar, statt aus einer Liste gefiltert zu werden. Ein Test importiert das
Servermodul in einem frischen Interpreter und prüft
`sys.modules`.[^s-mcp-read-server-implementation-session-2026-09-02]
- **Zwei Transports.** `stdio` zum Entwickeln und Testen ohne Netz; `streamable-http` für die
Auslieferung, der einzige, vor den sich ein HTTP-Reverse-Proxy setzen kann. `sse` ist über das
SDK erreichbar und wird bewusst nicht angeboten - der abgelöste Remote-Transport, jetzt darauf
zu bauen verschiebt den Wechsel nur.
- **Jede Antwort trägt den Commit, aus dem sie berechnet wurde** (`commit`, `as_of`). Ein
veralteter Checkout antwortet sonst selbstbewusst falsch. `null` heißt: der bediente Baum hat
uncommittete Änderungen, die Antwort entspricht keiner Revision. Der Stempel ist die Revision,
aus der die Seiten *tatsächlich* gelesen wurden, nicht die zum Zeitpunkt des Stempelns aktuelle
- ein Bug, der genau diesen Unterschied überging, wurde beim Schreiben des Golden-Tests selbst
gefunden und behoben.[^s-mcp-read-server-implementation-session-2026-09-02]
- **Telemetrie in den bedienten Baum wird beim Start verweigert**, nicht still umgeleitet. Der
Sync, der den Checkout aktuell hält (`git fetch && git reset --hard`), darf `reports/telemetry/`
wegräumen; ein Trace, der dort landet, wäre ein Verlust und eine stille Möglichkeit, den Baum
zu beschmutzen, dessen Sauberkeit der Korpus-Cache prüft.
- **Kein Iteration Budget Gate im Server.** Das Gate begrenzt eine Agenten-Session am unbemerkten
Iterieren über den Wiki-Zustand, nicht einen Nutzer, der oft sucht - Retrieval ist deshalb
bereits generell davon ausgenommen (siehe [[Iteration and Cost Limits]]). Rate Limiting gehört
stattdessen vor den Prozess, neben die Authentifizierung.
- **Authentifizierung ist Middleware, nicht Servercode.** Eine Traefik-ForwardAuth-Instanz
(Bearer-Token gegen SHA-256-Hashes) sitzt vor dem Prozess; nicht sauber authentifizierte
Zugriffe erreichen Python gar nicht erst.
- **Gemessen:** Korpus-Parse für 176 Seiten 265 ms → 54 ms (`CSafeLoader`),
`wikitool search` end-to-end 593 ms → 347 ms; die verbleibenden ~262 ms sind Modulimport und
entfallen im residenten Serverprozess, weil er ihn einmal pro Start statt pro Aufruf
zahlt.[^s-mcp-read-server-implementation-session-2026-09-02]
## Beispiele
- `search`/`types`/`describe_type`/`lint`/`status` als die fünf Tools - siehe
`tools/chemenu/mcp/server.py`.
- Der Korpus-Cache (`chemenu/corpus_cache.py`) hält einen Parse pro Commit und cacht nie einen
schmutzigen Arbeitsbaum - dieselbe Eigenschaft, die den Antwort-Stempel korrekt hält.
- `chemenu.api.Corpus`: nimmt einen Root, liefert exakt die `--json`-Formen der CLI, raised statt
zu exitieren.
## Wann zu verwenden
- Ein Konsument, der keine Shell auf der bedienenden Maschine ist, soll dieselben Fragen stellen
können wie ein Agent, der `wikitool` direkt aufruft.
- Mehrere gleichzeitige Leser eines Korpus, für die ein Prozess pro CLI-Aufruf (Modulimport,
Korpus-Parse) unnötigen Overhead bedeutet.
## Wann NICHT zu verwenden
- Als Ort für einen Schreibpfad - die Ingest-Queue (geplant, Issue #32) ist ein anderes Design
mit einer Quarantäne davor, nicht eine Erweiterung dieses Servers.
- Als Ersatz für den Iteration Budget Gate oder das Traefik-Rate-Limiting - beide bleiben
notwendig und leben an anderer Stelle.
## Verwandte Concepts
- [[Publish-Remote Gate]]
- [[Mass-Update Gate]]
- [[Iteration and Cost Limits]]
## Fußnoten
[^s-mcp-read-server-implementation-session-2026-09-02]: [[Source - MCP Read Server Implementation Session 2026-09-02]]
## Beziehungen
- **implementiert:** [[wikitool]]
- **verwandt mit:** [[Publish-Remote Gate]]
- **verwandt mit:** [[Mass-Update Gate]]
- **ausgenommen von:** [[Iteration and Cost Limits]]
- **Teil von:** [[Chemenu]]
## Siehe auch
- [[wikitool]]
- [[Publish-Remote Gate]]
- [[Mass-Update Gate]]
- [[Iteration and Cost Limits]]
- [[Chemenu]]
- [[Source - MCP Read Server Implementation Session 2026-09-02]]
+5 -1
View File
@@ -4,7 +4,7 @@ concept_type: workflow
tags: [gate, safety, mass-update, confirmation] tags: [gate, safety, mass-update, confirmation]
created: 2026-08-03 created: 2026-08-03
modified: 2026-09-01 modified: 2026-09-01
related: [Content Quality Control, wikitool, Iteration and Cost Limits, Structural Enforcement over Documented Rule, Bulk Operations] related: [Content Quality Control, wikitool, Iteration and Cost Limits, Structural Enforcement over Documented Rule, Bulk Operations, Publish-Remote Gate, MCP-Leseserver]
sources: [Source - LLM Improvements Sonnet Analysis, Source - LLM Improvements Production Agent Gaps 2026, Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31, Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31, Source - LLM Improvements Codex Analysis] sources: [Source - LLM Improvements Sonnet Analysis, Source - LLM Improvements Production Agent Gaps 2026, Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31, Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31, Source - LLM Improvements Codex Analysis]
confidence: 0.88 confidence: 0.88
confidence_base: 0.88 confidence_base: 0.88
@@ -77,6 +77,8 @@ Das Mass-Update Gate ist ein Sicherheitsmechanismus, der die Ausführung pausier
- **wird gespiegelt durch:** [[Iteration and Cost Limits]] - **wird gespiegelt durch:** [[Iteration and Cost Limits]]
- **wendet an:** [[Structural Enforcement over Documented Rule]] - **wendet an:** [[Structural Enforcement over Documented Rule]]
- **grenzt ab gegen:** [[Bulk Operations]] - **grenzt ab gegen:** [[Bulk Operations]]
- **verwandtes Gate:** [[Publish-Remote Gate]]
- **verwandt mit:** [[MCP-Leseserver]]
## Siehe auch ## Siehe auch
@@ -86,6 +88,8 @@ Das Mass-Update Gate ist ein Sicherheitsmechanismus, der die Ausführung pausier
- [[Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31]] - [[Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31]]
- [[Structural Enforcement over Documented Rule]] - [[Structural Enforcement over Documented Rule]]
- [[Bulk Operations]] - [[Bulk Operations]]
- [[Publish-Remote Gate]]
- [[MCP-Leseserver]]
## Fußnoten ## Fußnoten
+116
View File
@@ -0,0 +1,116 @@
---
type: types/concept.md
concept_type: workflow
tags: []
created: 2026-09-01
modified: 2026-09-02
related: [Mass-Update Gate, Chemenu, MCP-Leseserver]
sources: [Source - Publish-Remote Gate and Issue Triage Session 2026-09-01, Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01, Source - MCP Read Server Implementation Session 2026-09-02]
confidence: 0.70
confidence_base: 0.70
provenance: sourced
summary: 'Drittes, im Code durchgesetztes Gate: publish bricht mit Exit 42 ab, wenn die aufgeloeste Push-URL nicht in einer optionalen, gitignoreten Allowlist steht; doctor benennt seit 2026-09-02 den Gate-Zustand statt nur die Dateiexistenz'
---
# Publish-Remote Gate
**Typ:** Workflow
## Definition
Ein im Code durchgesetztes Gate, das einen Schreibvorgang (hier: `publish`) auf ein
deklariertes, per Datei zugelassenes Ziel beschränkt. Der Vorgang bricht ab, wenn das
aufgelöste Push-Ziel nicht in der Allowlist steht - unabhängig davon, unter welchem Namen der
Remote lokal konfiguriert ist.
## Kernpunkte
- **Geprüft wird die aufgelöste URL, nicht der Remote-Name.** Ein namensbasiertes Gate würde
ein `publish` durchlassen, dessen `origin` zwischenzeitlich auf ein anderes Ziel umgebogen
wurde - genau der Fall, den das Gate abfangen soll.
- **Kein Freigabe-Token, anders als vergleichbare Gates.** Ein Gate, dessen Frage per
Änderungssatz beantwortbar ist ("ist diese konkrete Änderung richtig?"), kann sich mit einem
Token lösen, den ein Mensch einmalig ausstellt. Ein Gate, dessen Frage eine stehende
Eigenschaft des Checkouts ist ("gehört dieser Inhalt grundsätzlich in dieses Ziel?"), sollte
keinen Token haben - der einzige Weg daran vorbei ist ein bewusster Edit der Konfigurationsdatei
durch den Menschen, nie ein automatisierter Bypass.
- **Die Allowlist-Datei ist per Checkout, nicht Teil des versionierten Inhalts.** Sie
beschreibt, wohin *dieser* Checkout schreiben darf - eine committete Kopie würde jedem Klon
dieselbe Erlaubnis unterschieben, unabhängig davon, ob sie für ihn zutrifft.
- **Fehlende Datei bedeutet unbeschränkt, kaputte Datei bedeutet Fehler.** Diese Unterscheidung
ist wichtig: Ein Checkout ohne Beschränkungsbedarf soll nicht gezwungen sein, eine leere
Konfigurationsdatei zu pflegen; eine beschädigte Datei darf aber nicht wie eine abwesende
behandelt werden, sonst wird eine defekte Sicherung zu einer stillschweigend abgeschalteten.
- **`doctor` benennt seit 2026-09-02 den Gate-*Zustand*, nicht nur, ob die Datei existiert.**
Vorher meldete der Check nur die Anwesenheit von `.wikitool-remotes.json`; ob das
gleichbedeutend mit "scharf" ist, musste der Leser selbst schließen. Alle drei Ausgaben
beginnen jetzt mit `Gate armed:` bzw. `Gate not armed:` - der Ein-Remote-Fall ohne Allowlist
bleibt `OK` (er hat nichts zu schützen), sagt aber ausdrücklich, dass jedes Push-Ziel
durchkommt.[^s-mcp-read-server-implementation-session-2026-09-02]
## Wann zu verwenden
- Ein Checkout kann an mehr als ein Remote-Ziel schreiben, und ein Schreibvorgang an das
falsche Ziel ist teuer oder nicht rückgängig zu machen (z. B. weil das Ziel öffentlich ist).
- Die Menge der zulässigen Ziele ist eine stabile Eigenschaft des Checkouts, keine
Einzelfallentscheidung pro Vorgang.
## Wann NICHT zu verwenden
- Wenn nur ein Remote existiert und kein Risiko einer Zielverwechslung besteht - dort ist die
Allowlist reine Formalität ohne Schutzwirkung.
- Für Entscheidungen, die tatsächlich pro Änderungssatz getroffen werden sollen (dafür ist ein
Token-basiertes Gate wie das Mass-Update-Gate das richtige Muster).
## Was das Gate nicht abdeckt: Inhalt, der über einen Merge hereinkommt
Das Gate schützt den **Push**, nicht den **Merge**. Ein Setup, bei dem eine private Instanz
Maschinerie von einem öffentlichen Upstream per `git merge upstream/main` zieht, hat ein
eigenes, empirisch geprüftes Problem: Ein einfacher Merge übernimmt Upstream-Änderungen an
bereits gelöschten Inhaltsseiten nicht sauber.
Gemessen an einem Wegwerf-Repo-Paar, bei dem der Upstream nach der einmaligen Löschung des
Demo-Korpus eine Seite ändert, eine neue anlegt und eine dritte löscht:
- Eine **geänderte** Seite erzeugt einen `modify/delete`-Konflikt und lässt die
Upstream-Fassung im Arbeitsbaum liegen - ein naives Auflösen mit `git add -A` holt sie zurück.
- Eine **neu angelegte** Seite wird **stillschweigend** übernommen, ohne Konflikt und ohne
Meldung.
- Eine beidseitig gelöschte Seite verursacht nichts - der einzige Fall, der ohne Weiteres
funktioniert.
Ein naheliegender Fix (`.gitattributes` mit `merge=ours` für die betroffenen Verzeichnisse)
wurde ebenfalls gemessen und verworfen: Der Treiber wirkt nur bei Inhaltskonflikten auf
beidseitig vorhandenen Dateien, nicht bei modify/delete-Paaren oder Neuanlagen.
Die funktionierende Prozedur hält den Merge mit `--no-commit` offen, erzwingt die
Inhaltsverzeichnisse zurück auf den Stand vor dem Merge, solange `HEAD` noch dorthin zeigt, und
prüft danach explizit (`git diff --name-only $BEFORE HEAD -- kb raw` muss leer sein) - eine
Kontrolle, die nicht stillschweigend übersprungen werden kann, anders als eine bloße Behauptung
im Text[^s-private-instance-merge-correction-and-issue-30-session-2026-09-01]. Details, ein
getestetes Skript und zwei Architekturvorschläge (das Verfahren als `wikitool`-Kommando bauen,
oder den Demo-Korpus grundsätzlich von dem Branch fernhalten, von dem private Instanzen ihre
Maschinerie ziehen) stehen in Gitea-Issue #30.
## Verwandte Concepts
- [[Mass-Update Gate]]
- [[MCP-Leseserver]]
## Beziehungen
- **gilt fuer:** [[Chemenu]]
- **verwandtes Gate:** [[Mass-Update Gate]]
- **verwandt mit:** [[MCP-Leseserver]]
## Siehe auch
- [[Source - Publish-Remote Gate and Issue Triage Session 2026-09-01]]
- [[Chemenu]]
- [[Mass-Update Gate]]
- [[Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01]]
- [[MCP-Leseserver]]
## Fußnoten
[^s-private-instance-merge-correction-and-issue-30-session-2026-09-01]: [[Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01]]
[^s-mcp-read-server-implementation-session-2026-09-02]: [[Source - MCP Read Server Implementation Session 2026-09-02]]
+25 -2
View File
@@ -1,3 +1,13 @@
---
profile: entities
outbound:
entities: [depends-on, required-by, runs-on, hosts, uses, produces, consumes, maintains, owns, part-of, composition, supersedes, see-also]
concepts: [implements, exemplifies, rests-on, applies-when, operates-on, invokes, see-also]
sources: [evidenced-by, defined-in, see-also]
comparisons: [compares-with, see-also]
required_by_stack: false
---
# kb/entities/ - Collection Contract # kb/entities/ - Collection Contract
Concrete things that exist: a project, a deployed system, a CLI tool, a technology, a person or Concrete things that exist: a project, a deployed system, a CLI tool, a technology, a person or
@@ -7,8 +17,10 @@ an organization. If it can be pointed at, it is an entity.
is, where it actually is, and whether that is still true. An entity page that describes a is, where it actually is, and whether that is still true. An entity page that describes a
system correctly but names no host, path, version or status has not earned its keep. system correctly but names no host, path, version or status has not earned its keep.
Inherits [kb/CONTRACT.md](../CONTRACT.md) - naming, tone, linking, provenance and confidence Inherits [kb/CONTRACT.md](../CONTRACT.md) for the rules the stack enforces - linking mechanics,
are defined there and are not restated here. provenance, citation, the confidence machinery - and
[kb/CONVENTIONS.md](../CONVENTIONS.md) for what this instance decided: language, naming forms,
tone, relationship labels, the confidence rubric. Neither is restated here.
## Types offered ## Types offered
@@ -36,6 +48,17 @@ These are areas, not collections: they inherit this contract and carry no `COLLE
- **People** - role, affiliation, and the projects or decisions they are connected to. Nothing - **People** - role, affiliation, and the projects or decisions they are connected to. Nothing
personal beyond what the source states. personal beyond what the source states.
## Authorised labels
The `outbound:` block above is what `wikitool lint` and `xref add` check: which labels a page in
this collection may use, per destination. The catalogue they are drawn from - and what each one
asserts - is [instructions/link-taxonomy.md](../../instructions/link-taxonomy.md), which binds
nothing on its own.
Operational labels dominate here because an entity's relationships are mostly to other concrete things. `implements` points *out* to a concept; the concept does not point back unless that direction is a statement of its own.
Adding a label here is a deliberate contract change, not a way around a refusal.
## Outbound linking ## Outbound linking
An entity links to the technologies it uses, the systems it runs on, the projects that depend An entity links to the technologies it uses, the systems it runs on, the projects that depend
+3 -3
View File
@@ -19,7 +19,7 @@
|------|------|---------|----------------| |------|------|---------|----------------|
| [[andybalholm-edl]] | project | Go-basierte EDL-Bibliothek für die Kommunikation mit eingebetteten Geräten. | 2026-08-29 | | [[andybalholm-edl]] | project | Go-basierte EDL-Bibliothek für die Kommunikation mit eingebetteten Geräten. | 2026-08-29 |
| [[BCDModule]] | project | Go-Modul, das die Entscheidungslogik für die Batterieladung umsetzt. | 2026-08-29 | | [[BCDModule]] | project | Go-Modul, das die Entscheidungslogik für die Batterieladung umsetzt. | 2026-08-29 |
| [[Chemenu]] | project | Persoenliches IT-Wissenswiki, gepflegt von LLM-Agenten; seit 1.0.0 versioniert, seit 1.1.0 Personalization Plane, seit 1.8.0 ENVIRONMENT.md, seit 2.0.0 unter dem Namen Chemenu (vorher llm-wiki-test1) mit dem Python-Paket chemenu | 2026-09-01 | | [[Chemenu]] | project | Deterministischer Wissenskompiler (raw/ -> kb/); seit 2.0.0 unter dem Namen Chemenu; Issue 26 zur Versionsstellen-Nomenklatur in 2.5.0 geschlossen | 2026-09-02 |
| [[goresponsiveness]] | project | Go-Werkzeug zur Messung von Anwendungsleistung und Responsiveness. | 2026-08-29 | | [[goresponsiveness]] | project | Go-Werkzeug zur Messung von Anwendungsleistung und Responsiveness. | 2026-08-29 |
| [[ha-core]] | project | Kern-Integrationsbibliothek für Home-Assistant-E3DC-Systeme; stellt die E3DC-Kommunikationsprotokolle und den Home-Assistant-Integrationscode bereit. | 2026-08-29 | | [[ha-core]] | project | Kern-Integrationsbibliothek für Home-Assistant-E3DC-Systeme; stellt die E3DC-Kommunikationsprotokolle und den Home-Assistant-Integrationscode bereit. | 2026-08-29 |
| [[hacs-e3dc]] | project | Home Assistant Custom Component zur Überwachung von E3DC-Energiesystemen. | 2026-08-29 | | [[hacs-e3dc]] | project | Home Assistant Custom Component zur Überwachung von E3DC-Energiesystemen. | 2026-08-29 |
@@ -96,8 +96,8 @@
| [[pascalandy schema]] | tool | Von der Community beigesteuertes Wiki Schema (Global) aus pascalandys Kommentar in Farzas Gist, mit alternativer Tag-Taxonomie (area/kind/topic/status/pty) | 2026-08-29 | | [[pascalandy schema]] | tool | Von der Community beigesteuertes Wiki Schema (Global) aus pascalandys Kommentar in Farzas Gist, mit alternativer Tag-Taxonomie (area/kind/topic/status/pty) | 2026-08-29 |
| [[Pi]] | tool | Assistenz-Agent von Inflection AI; kann das LLM-Wiki-Muster wie andere LLM-Agenten umsetzen. | 2026-08-29 | | [[Pi]] | tool | Assistenz-Agent von Inflection AI; kann das LLM-Wiki-Muster wie andere LLM-Agenten umsetzen. | 2026-08-29 |
| [[Proton]] | tool | Wine-basierte Kompatibilitätsschicht von Valve; lässt Windows-Spiele über Steam unter Linux laufen, mit optimierter DirectX-Übersetzung. | 2026-08-29 | | [[Proton]] | tool | Wine-basierte Kompatibilitätsschicht von Valve; lässt Windows-Spiele über Steam unter Linux laufen, mit optimierter DirectX-Übersetzung. | 2026-08-29 |
| [[qmd]] | tool | Lokale Suchmaschine für Markdown-Dateien mit hybrider BM25-Vektor-Suche und LLM-Reranking. | 2026-08-29 | | [[qmd]] | tool | Lokale Suchmaschine fuer Markdown-Dateien: TypeScript/Node.js/Bun, SQLite-FTS5-BM25 plus sqlite-vec-Vektorsuche plus node-llama-cpp-LLM-Reranking. | 2026-09-02 |
| [[Steam]] | tool | Valves Plattform für digitalen Spielevertrieb und Spielebibliothek auf dem PC. | 2026-08-29 | | [[Steam]] | tool | Valves Plattform für digitalen Spielevertrieb und Spielebibliothek auf dem PC. | 2026-08-29 |
| [[wikitool]] | tool | Deterministisches CLI fuer alle mechanischen Wiki-Operationen; seit 2.0.0 liegt es im Python-Paket chemenu, das Kommando heisst weiterhin wikitool | 2026-09-01 | | [[wikitool]] | tool | Deterministisches CLI fuer alle mechanischen Wiki-Operationen; seit 2.0.0 im Paket chemenu, seit 2.4.0 zusaetzlich als MCP-Leseserver erreichbar, seit 2.5.0 mit Breaking-Change-Pflichtmarker | 2026-09-02 |
| [[Wine]] | tool | Kompatibilitätsschicht, die Windows-API-Aufrufe nach POSIX übersetzt und Windows-Anwendungen unter Linux, BSD und macOS ohne Virtualisierung oder Emulation ausführt. | 2026-08-29 | | [[Wine]] | tool | Kompatibilitätsschicht, die Windows-API-Aufrufe nach POSIX übersetzt und Windows-Anwendungen unter Linux, BSD und macOS ohne Virtualisierung oder Emulation ausführt. | 2026-08-29 |
+43 -7
View File
@@ -3,13 +3,13 @@ type: types/entity.md
entity_type: project entity_type: project
tags: [wiki, llm, knowledge-base] tags: [wiki, llm, knowledge-base]
created: 2026-08-04 created: 2026-08-04
modified: 2026-09-01 modified: 2026-09-02
related: [Personalization Plane, Issue Label Scheme, Optional Instance Context File] related: [Personalization Plane, Issue Label Scheme, Optional Instance Context File, Delete Rather Than Anonymize, Dual Licensing by File Plan, Publish-Remote Gate, MCP-Leseserver]
sources: [Source - Copilot Skill Restructure Instructions, Source - Conversation - AGENTS.md Skill Restructuring Session 2026-08-04, Source - Conversation - Versioning CI-CD and Content Migration Session 2026-08-30, Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31, Source - Conversation - Issue Triage Labels and TODO Retirement Session 2026-08-31, Source - Conversation - Write-Once Frontmatter Fields and touch --set Session 2026-08-31, Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31, Source - Conversation - Two Round-Trip Defects Found by an Ingest Session 2026-08-31, Source - Conversation - Hardening the Test Suite Against Silent Environment Dependencies Session 2026-08-31] sources: [Source - Copilot Skill Restructure Instructions, Source - Conversation - AGENTS.md Skill Restructuring Session 2026-08-04, Source - Conversation - Versioning CI-CD and Content Migration Session 2026-08-30, Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31, Source - Conversation - Issue Triage Labels and TODO Retirement Session 2026-08-31, Source - Conversation - Write-Once Frontmatter Fields and touch --set Session 2026-08-31, Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31, Source - Conversation - Two Round-Trip Defects Found by an Ingest Session 2026-08-31, Source - Conversation - Hardening the Test Suite Against Silent Environment Dependencies Session 2026-08-31, 'Source - Public Release, Corpus Purge and History Squash Session 2026-09-01', Source - Publish-Remote Gate and Issue Triage Session 2026-09-01, Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01, Source - MCP Read Server Implementation Session 2026-09-02, Source - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02]
confidence: 0.90 confidence: 0.90
confidence_base: 0.90 confidence_base: 0.90
provenance: mixed provenance: mixed
summary: Persoenliches IT-Wissenswiki, gepflegt von LLM-Agenten; seit 1.0.0 versioniert, seit 1.1.0 Personalization Plane, seit 1.8.0 ENVIRONMENT.md, seit 2.0.0 unter dem Namen Chemenu (vorher llm-wiki-test1) mit dem Python-Paket chemenu summary: Deterministischer Wissenskompiler (raw/ -> kb/); seit 2.0.0 unter dem Namen Chemenu; Issue 26 zur Versionsstellen-Nomenklatur in 2.5.0 geschlossen
--- ---
# Chemenu # Chemenu
@@ -17,7 +17,21 @@ summary: Persoenliches IT-Wissenswiki, gepflegt von LLM-Agenten; seit 1.0.0 vers
## Beschreibung ## Beschreibung
Chemenu ist das persönliche IT-Wissens-Wiki-Repository, das Gegenstand der Skill-Umstrukturierung ist, die in den Copilot Skill Restructure Instructions beschrieben wird[^s-copilot-skill-restructure-instructions]. Es hat derzeit eine monolithische `AGENTS.md`-Datei (~30KB), die fünf Workflows (INGEST, QUERY, LINT, CREATE, UPDATE), das Frontmatter-Schema des Wikis, Entity/Concept-Typ-Tabellen, Provenance/Citation-Regeln, Naming-Konventionen und eine Konfidenz-Scoring-Formel beschreibt[^s-copilot-skill-restructure-instructions]. Chemenu ist ein deterministischer Wissenskompiler: Rohnotizen unter `raw/` werden über eine
Schema- und Werkzeugschicht zu einem verlinkten, quellengebundenen Wiki unter `kb/` verdichtet.
Was mechanisch ist, macht `tools/wikitool`; was Urteil braucht, macht ein Agent unter Contracts,
deren Grenzen in Code durchgesetzt sind statt im Prompt.
**Diese Instanz ist seit dem 2026-09-01 öffentlich** und dient als Testbett und Demo, nicht als
produktive Wissensbasis. Ihr Korpus beschreibt überwiegend den Stack selbst - Gates, Lint,
Versionierung, Suche, das Wiki-Muster -, sodass die Distribution sich mit ihren eigenen Mitteln
dokumentiert. Lizenz: AGPL-3.0 für den Stack, CC-BY-4.0 für die Inhalte.
**Ausgangspunkt war eine monolithische Struktur.** Zum Zeitpunkt der Copilot Skill Restructure
Instructions lag alles in einer einzigen `AGENTS.md` (~30 KB): fünf Workflows (INGEST, QUERY,
LINT, CREATE, UPDATE), das Frontmatter-Schema, Entity- und Concept-Typtabellen,
Provenance- und Zitatregeln, Namenskonventionen und die Konfidenz-Formel[^s-copilot-skill-restructure-instructions].
Die Aufteilung in Skills und Contracts, die daraus folgte, ist seit 2026-08-04 abgeschlossen.
**Der Name.** Bis zum 2026-09-01 hieß das Projekt `llm-wiki-test1` - ein Arbeitstitel mit einer **Der Name.** Bis zum 2026-09-01 hieß das Projekt `llm-wiki-test1` - ein Arbeitstitel mit einer
Ordnungszahl darin, kein Name. **Chemenu** ist die deutsche Wikipedia-Schreibweise des Ordnungszahl darin, kein Name. **Chemenu** ist die deutsche Wikipedia-Schreibweise des
@@ -30,9 +44,10 @@ Das Repository hat bereits ein deterministisches CLI, `tools/wikitool` (Python,
## Kerndaten ## Kerndaten
- **Zweck:** Persönliche IT-Wissensbasis - **Zweck:** Deterministischer Wissenskompiler; diese Instanz ist Testbett und öffentliche Demo
- **Status:** Aktiv - Skill-Umstrukturierung abgeschlossen 2026-08-04 - **Status:** Aktiv - Skill-Umstrukturierung abgeschlossen 2026-08-04
- **Verantwortlich:** Torben - **Verantwortlich:** Torben
- **Lizenz:** AGPL-3.0 (Stack: `tools/`, `types/`), CC-BY-4.0 (Inhalte)
- **Repository:** `torben/chemenu` auf gitea.nehmer.net; bis 2026-09-01 `torben/llm-wiki-test1` - **Repository:** `torben/chemenu` auf gitea.nehmer.net; bis 2026-09-01 `torben/llm-wiki-test1`
- **Architektur:** Dreilagig: raw/ (Quelle), wiki/ (Wissen), tools/ (deterministisches CLI) - **Architektur:** Dreilagig: raw/ (Quelle), wiki/ (Wissen), tools/ (deterministisches CLI)
@@ -45,6 +60,10 @@ Das Repository hat bereits ein deterministisches CLI, `tools/wikitool` (Python,
- **verwendet:** [[Personalization Plane]] - **verwendet:** [[Personalization Plane]]
- **verwendet:** [[Issue Label Scheme]] - **verwendet:** [[Issue Label Scheme]]
- **verwendet:** [[Optional Instance Context File]] - **verwendet:** [[Optional Instance Context File]]
- **wendet an:** [[Delete Rather Than Anonymize]]
- **wendet an:** [[Dual Licensing by File Plan]]
- **setzt um:** [[Publish-Remote Gate]]
- **enthaelt:** [[MCP-Leseserver]]
## Details ## Details
@@ -124,6 +143,13 @@ Repositorys selbst und keine Aussagen aus einer Rohdatenquelle; der Änderungsda
## Historie ## Historie
- 2026-09-02 - `2.2.3`-`2.4.1`: Issue #36s vierstufige Sequenz zum [[MCP-Leseserver]] umgesetzt
([[Publish-Remote Gate]] scharf gestellt, Lesepfad gehärtet, Root-Auflösung von der Importzeit
gelöst, der Server selbst), dazu `INSTALL-MCP.md` und Issue #37 für das noch fehlende
Container-Image. Alle vier Sequenz-Issues geschlossen; #36 blieb offen, weil sein eigenes
Abschlusskriterium - ein Konsument, der nachweislich über die
Authentifizierungs-Middleware antwortet - erst mit #37 erfüllbar
ist.[^s-mcp-read-server-implementation-session-2026-09-02]
- 2026-09-01 - `2.0.0` (Commit `9a7abe6`, 121 Dateien, 730 Tests grün): Rebranding von - 2026-09-01 - `2.0.0` (Commit `9a7abe6`, 121 Dateien, 730 Tests grün): Rebranding von
`llm-wiki-test1` auf **Chemenu** nach Gitea-Issue #3 - Repo-Rename, Produktname, `llm-wiki-test1` auf **Chemenu** nach Gitea-Issue #3 - Repo-Rename, Produktname,
Release-Artefakt (`chemenu-stack-<version>.tar.gz`), Release-Feed, und das Python-Paket Release-Artefakt (`chemenu-stack-<version>.tar.gz`), Release-Feed, und das Python-Paket
@@ -133,7 +159,8 @@ Repositorys selbst und keine Aussagen aus einer Rohdatenquelle; der Änderungsda
auf den alten Repo-Pfad und lässt sich per Invariante 1 nicht von Hand reparieren. Die auf den alten Repo-Pfad und lässt sich per Invariante 1 nicht von Hand reparieren. Die
erste Einschätzung lautete `1.9.0` und wurde von Torben korrigiert; die Lücke in der Doku, erste Einschätzung lautete `1.9.0` und wurde von Torben korrigiert; die Lücke in der Doku,
die dazu führte - MAJOR ist dort als Inhaltsmigration statt als Kompatibilitätsbruch die dazu führte - MAJOR ist dort als Inhaltsmigration statt als Kompatibilitätsbruch
beschrieben - liegt als Issue #26. Verzeichnet in `CHANGES.md` (`2.0.0`). beschrieben - war Issue #26 und wurde in `2.5.0` geschlossen (siehe [[KB Stack Versioning]]).
Verzeichnet in `CHANGES.md` (`2.0.0`).
- 2026-08-31 - `1.8.1` (Commit `a243a4a`, Korrektur `2b7b3cb`): Test-Coverage wird in CI - 2026-08-31 - `1.8.1` (Commit `a243a4a`, Korrektur `2b7b3cb`): Test-Coverage wird in CI
gemessen und als Artefakt ausgewiesen, ohne `--cov-fail-under` - siehe gemessen und als Artefakt ausgewiesen, ohne `--cov-fail-under` - siehe
Messen vor Schwelle. Die Messung deckte einen `dist export`-Fehler auf: Coverage-Ausgabe Messen vor Schwelle. Die Messung deckte einen `dist export`-Fehler auf: Coverage-Ausgabe
@@ -195,6 +222,14 @@ Repositorys selbst und keine Aussagen aus einer Rohdatenquelle; der Änderungsda
- [[Source - Conversation - Two Round-Trip Defects Found by an Ingest Session 2026-08-31]] - [[Source - Conversation - Two Round-Trip Defects Found by an Ingest Session 2026-08-31]]
- [[Source - Conversation - Hardening the Test Suite Against Silent Environment Dependencies Session 2026-08-31]] - [[Source - Conversation - Hardening the Test Suite Against Silent Environment Dependencies Session 2026-08-31]]
- [[Optional Instance Context File]] - [[Optional Instance Context File]]
- [[Source - Public Release, Corpus Purge and History Squash Session 2026-09-01]]
- [[Delete Rather Than Anonymize]]
- [[Dual Licensing by File Plan]]
- [[Source - Publish-Remote Gate and Issue Triage Session 2026-09-01]]
- [[Publish-Remote Gate]]
- [[Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01]]
- [[MCP-Leseserver]]
- [[Source - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02]]
## Fußnoten ## Fußnoten
@@ -206,3 +241,4 @@ Repositorys selbst und keine Aussagen aus einer Rohdatenquelle; der Änderungsda
[^s-conversation-gate-counting-and-measured-calibration-session-2026-08-31]: [[Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31]] [^s-conversation-gate-counting-and-measured-calibration-session-2026-08-31]: [[Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31]]
[^s-conversation-write-once-frontmatter-fields-and-touch-set-session-2026-08-31]: [[Source - Conversation - Write-Once Frontmatter Fields and touch --set Session 2026-08-31]] [^s-conversation-write-once-frontmatter-fields-and-touch-set-session-2026-08-31]: [[Source - Conversation - Write-Once Frontmatter Fields and touch --set Session 2026-08-31]]
[^s-conversation-comma-bug-budget-refund-and-lint-report-path-session-2026-08-31]: [[Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31]] [^s-conversation-comma-bug-budget-refund-and-lint-report-path-session-2026-08-31]: [[Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31]]
[^s-mcp-read-server-implementation-session-2026-09-02]: [[Source - MCP Read Server Implementation Session 2026-09-02]]
+22 -11
View File
@@ -3,13 +3,13 @@ type: types/entity.md
entity_type: tool entity_type: tool
tags: [search, markdown, cli, local] tags: [search, markdown, cli, local]
created: 2026-07-26 created: 2026-07-26
modified: 2026-08-29 modified: 2026-09-02
related: [Obsidian, LLM Wiki Pattern] related: [Obsidian, LLM Wiki Pattern]
sources: [Source - LLM Wiki Pattern] sources: [Source - LLM Wiki Pattern, Source - qmd - GitHub Repository]
confidence: 0.85 confidence: 0.80
confidence_base: 0.85 confidence_base: 0.80
provenance: sourced provenance: sourced
summary: Lokale Suchmaschine für Markdown-Dateien mit hybrider BM25-Vektor-Suche und LLM-Reranking. summary: 'Lokale Suchmaschine fuer Markdown-Dateien: TypeScript/Node.js/Bun, SQLite-FTS5-BM25 plus sqlite-vec-Vektorsuche plus node-llama-cpp-LLM-Reranking.'
--- ---
# qmd # qmd
@@ -22,11 +22,12 @@ qmd ist eine lokale Suchmaschine, die speziell für Markdown-Dateien ausgelegt i
## Kerndaten ## Kerndaten
- **Typ:** CLI-Tool - **Typ:** CLI-Tool
- **Sprache:** Nicht angegeben (wahrscheinlich Go oder Rust) - **Sprache:** TypeScript, Laufzeit Node.js oder Bun
- **Such-Typen:** Hybrid (BM25 + Vector) - **Such-Typen:** Hybrid (SQLite-FTS5-BM25 + `sqlite-vec`-Vektorsuche)
- **Neu-Ranking:** LLM-basiert - **Neu-Ranking:** LLM-basiert über `node-llama-cpp` (lokale GGUF-Modelle)
- **Bereitstellung:** On-device/lokal - **Bereitstellung:** On-device/lokal
- **Repository:** https://github.com/tobi/qmd - **Repository:** https://github.com/tobi/qmd
- **Installation:** `npm install -g @tobilu/qmd` oder `bun install -g @tobilu/qmd`
## Beziehungen ## Beziehungen
@@ -50,11 +51,13 @@ qmd ist eine lokale Suchmaschine, die speziell für Markdown-Dateien ausgelegt i
## Installation und Verwendung ## Installation und Verwendung
```bash ```bash
# Installation (hypothetisch, siehe aktuelles Repo für Details) # Installation (Node oder Bun)
go install github.com/tobi/qmd@latest npm install -g @tobilu/qmd
# oder
bun install -g @tobilu/qmd
# Suche von CLI # Suche von CLI
qmd search "knowledge management" qmd query "knowledge management"
# Als MCP-Server für LLM-Integration verwenden # Als MCP-Server für LLM-Integration verwenden
qmd server qmd server
@@ -74,9 +77,17 @@ qmd server
## Historie ## Historie
- [2026-09-02] - Sprachangabe korrigiert. Die Seite trug seit ihrer Erstellung "Nicht angegeben
(wahrscheinlich Go oder Rust)" bei `confidence: 0.85` - eine Vermutung mit einer Konfidenz, die
für eine Vermutung zu hoch war. Direkt gegen das Repository geprüft: TypeScript, Laufzeit
Node.js/Bun, SQLite-FTS5-BM25 + `sqlite-vec` + `node-llama-cpp`.[^s-qmd-github-repository]
- [2026-07-26] - Entity-Seite während der Aufnahme des LLM-Wiki-Pattern-Artikels erstellt - [2026-07-26] - Entity-Seite während der Aufnahme des LLM-Wiki-Pattern-Artikels erstellt
## Siehe auch ## Siehe auch
- [[LLM Wiki Pattern]] - [[LLM Wiki Pattern]]
- [[Obsidian]] - [[Obsidian]]
## Fußnoten
[^s-qmd-github-repository]: [[Source - qmd - GitHub Repository]]
+69 -6
View File
@@ -3,13 +3,13 @@ type: types/entity.md
entity_type: tool entity_type: tool
tags: [cli, automation, deterministic, wiki-management] tags: [cli, automation, deterministic, wiki-management]
created: 2026-08-03 created: 2026-08-03
modified: 2026-09-01 modified: 2026-09-02
related: [Semantic Lint Automation, Session Orientation, Iteration and Cost Limits, KB Stack Versioning, KB Migration, Personalization Plane, Detect-Repair Asymmetry, Write-Once Frontmatter Fields, Denylist over Allowlist, Command Round-Trip Integrity, Green Suite Blind Spot, Ambient Environment Dependency, Structural Enforcement over Documented Rule, Optional Instance Context File] related: [Semantic Lint Automation, Session Orientation, Iteration and Cost Limits, KB Stack Versioning, KB Migration, Personalization Plane, Detect-Repair Asymmetry, Write-Once Frontmatter Fields, Denylist over Allowlist, Command Round-Trip Integrity, Green Suite Blind Spot, Ambient Environment Dependency, Structural Enforcement over Documented Rule, Optional Instance Context File, MCP-Leseserver]
sources: [Source - LLM Improvements Codex Analysis, Source - LLM Improvements Sonnet Analysis, Source - Copilot Skill Restructure Instructions, Source - Conversation - AGENTS.md Skill Restructuring Session 2026-08-04, Source - LLM Improvements Production Agent Gaps 2026, Source - Conversation - Versioning CI-CD and Content Migration Session 2026-08-30, Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31, Source - Conversation - Issue Triage Labels and TODO Retirement Session 2026-08-31, Source - Conversation - Auto Mode and Tool Choice Session 2026-08-31, Source - Conversation - Write-Once Frontmatter Fields and touch --set Session 2026-08-31, Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31, Source - Conversation - Two Round-Trip Defects Found by an Ingest Session 2026-08-31, Source - Conversation - Hardening the Test Suite Against Silent Environment Dependencies Session 2026-08-31, Source - Conversation - ENVIRONMENT.md as an Optional Third Session-Level File Session 2026-08-31] sources: [Source - LLM Improvements Codex Analysis, Source - LLM Improvements Sonnet Analysis, Source - Copilot Skill Restructure Instructions, Source - Conversation - AGENTS.md Skill Restructuring Session 2026-08-04, Source - LLM Improvements Production Agent Gaps 2026, Source - Conversation - Versioning CI-CD and Content Migration Session 2026-08-30, Source - Conversation - Comma Bug Budget Refund and Lint Report Path Session 2026-08-31, Source - Conversation - Issue Triage Labels and TODO Retirement Session 2026-08-31, Source - Conversation - Auto Mode and Tool Choice Session 2026-08-31, Source - Conversation - Write-Once Frontmatter Fields and touch --set Session 2026-08-31, Source - Conversation - Gate Counting and Measured Calibration Session 2026-08-31, Source - Conversation - Two Round-Trip Defects Found by an Ingest Session 2026-08-31, Source - Conversation - Hardening the Test Suite Against Silent Environment Dependencies Session 2026-08-31, Source - Conversation - ENVIRONMENT.md as an Optional Third Session-Level File Session 2026-08-31, 'Source - Public Release, Corpus Purge and History Squash Session 2026-09-01', Source - Publish-Remote Gate and Issue Triage Session 2026-09-01, Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01, Source - MCP Read Server Implementation Session 2026-09-02, Source - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02]
confidence: 0.90 confidence: 0.90
confidence_base: 0.90 confidence_base: 0.90
provenance: sourced provenance: sourced
summary: Deterministisches CLI fuer alle mechanischen Wiki-Operationen; seit 2.0.0 liegt es im Python-Paket chemenu, das Kommando heisst weiterhin wikitool summary: Deterministisches CLI fuer alle mechanischen Wiki-Operationen; seit 2.0.0 im Paket chemenu, seit 2.4.0 zusaetzlich als MCP-Leseserver erreichbar, seit 2.5.0 mit Breaking-Change-Pflichtmarker
--- ---
# wikitool # wikitool
@@ -51,6 +51,7 @@ Wie in der Codex-Analyse vermerkt, bietet wikitool die deterministische Grundlag
- **zeigte:** [[Ambient Environment Dependency]] - **zeigte:** [[Ambient Environment Dependency]]
- **setzt um:** [[Structural Enforcement over Documented Rule]] - **setzt um:** [[Structural Enforcement over Documented Rule]]
- **setzt um:** [[Optional Instance Context File]] - **setzt um:** [[Optional Instance Context File]]
- **implementiert:** [[MCP-Leseserver]]
## Befehle ## Befehle
@@ -73,6 +74,9 @@ wikitool bietet die folgenden Befehlskategorien:
Seitenkopf zurückgefaltet statt abgelehnt, und weil der Block immer zuletzt gerendert wird, Seitenkopf zurückgefaltet statt abgelehnt, und weil der Block immer zuletzt gerendert wird,
richtet die erste Zitatoperation eine verrutschte Seite von selbst wieder ein[^s-conversation-two-round-trip-defects-found-by-an-ingest-session-2026-08-31] richtet die erste Zitatoperation eine verrutschte Seite von selbst wieder ein[^s-conversation-two-round-trip-defects-found-by-an-ingest-session-2026-08-31]
- **Abfrage:** `search` - Textsuche über `kb/` durch ein austauschbares Backend (`rg` heute), plus `--field`-Prädikate, die auf Frontmatter evaluiert werden (`entity_type=system`, `confidence>=0.8`, `tags=k8s`, `!source_url`). Ohne Text ist es eine reine strukturierte Abfrage. Schreibgeschützt und ausgenommen von der Iteration-Budget-Gate, da Abfrage das Lesen statt das Iterieren ist - **Abfrage:** `search` - Textsuche über `kb/` durch ein austauschbares Backend (`rg` heute), plus `--field`-Prädikate, die auf Frontmatter evaluiert werden (`entity_type=system`, `confidence>=0.8`, `tags=k8s`, `!source_url`). Ohne Text ist es eine reine strukturierte Abfrage. Schreibgeschützt und ausgenommen von der Iteration-Budget-Gate, da Abfrage das Lesen statt das Iterieren ist
- **MCP-Server:** `python -m chemenu.mcp` (`--transport stdio|streamable-http`) - kein
`wikitool`-Unterbefehl, sondern ein zweiter Einstiegspunkt über `chemenu.api.Corpus`, derselbe
Kern wie `search`/`types`/`lint`. Siehe [[MCP-Leseserver]]
- **Indizierung:** `index rebuild` - regeneriert die `kb/index.md`-Map plus eine pro-Sammlung `INDEX.md`, wobei ein Bereich bei 50 Zeilen in seine eigene Shard aufgeteilt wird - **Indizierung:** `index rebuild` - regeneriert die `kb/index.md`-Map plus eine pro-Sammlung `INDEX.md`, wobei ein Bereich bei 50 Zeilen in seine eigene Shard aufgeteilt wird
- **Herkunft:** `sources coverage`, `sources trace`, `sources rebuild-index` - **Herkunft:** `sources coverage`, `sources trace`, `sources rebuild-index`
- **Protokollierung:** `log append`, `log status` - **Protokollierung:** `log append`, `log status`
@@ -82,8 +86,15 @@ wikitool bietet die folgenden Befehlskategorien:
- **Veröffentlichung:** `publish` - zählt seit `1.5.0` nur noch Dateien, die eine Entscheidung tragen: Pfade unter `work/` und generierte Dateien (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, jede `INDEX.md`) werden committet und gepusht, aber nicht gegen die Schwelle gezählt; die Weigerungszeile weist beide Gründe getrennt aus[^s-conversation-gate-counting-and-measured-calibration-session-2026-08-31]. Siehe [[Mass-Update Gate]] - **Veröffentlichung:** `publish` - zählt seit `1.5.0` nur noch Dateien, die eine Entscheidung tragen: Pfade unter `work/` und generierte Dateien (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, jede `INDEX.md`) werden committet und gepusht, aber nicht gegen die Schwelle gezählt; die Weigerungszeile weist beide Gründe getrennt aus[^s-conversation-gate-counting-and-measured-calibration-session-2026-08-31]. Siehe [[Mass-Update Gate]]
- **Budget:** `budget status`, `budget reset` - Obergrenze seit `1.2.0` 60 Aufrufe je Sitzung; ein Aufruf, der über `_util.fail()` abgelehnt wurde, bekommt seinen Slot zurück und bleibt trotzdem in `recent`, damit der Loop-Breaker ihn sieht[^s-conversation-comma-bug-budget-refund-and-lint-report-path-session-2026-08-31]. Siehe [[Iteration and Cost Limits]] - **Budget:** `budget status`, `budget reset` - Obergrenze seit `1.2.0` 60 Aufrufe je Sitzung; ein Aufruf, der über `_util.fail()` abgelehnt wurde, bekommt seinen Slot zurück und bleibt trotzdem in `recent`, damit der Loop-Breaker ihn sieht[^s-conversation-comma-bug-budget-refund-and-lint-report-path-session-2026-08-31]. Siehe [[Iteration and Cost Limits]]
- **Versionierung:** `version bump`, `version check` - `bump` schreibt die Stack-Version in die - **Versionierung:** `version bump`, `version check` - `bump` schreibt die Stack-Version in die
Wurzeldatei `VERSION` und verweigert einen `MAJOR`-Sprung ohne Migrationsdokument, sofern er Wurzeldatei `VERSION`. Kompatibilität (ist die neue Version ein Drop-in-Ersatz - vorwärts ohne
nicht ausdrücklich mit `--no-migration "<Begründung>"` gesetzt wird; `check` ist der einzige Handarbeit, rückwärts noch downgradebar) und Inhaltsmigration sind seit `2.5.0` zwei getrennte
Fragen: ein Grenzübertritt verlangt zwingend `--breaking "<was aufhört zu funktionieren>"`,
verweigert auf jedem anderen Bump, und *zusätzlich* entweder ein Migrationsdokument oder
`--no-migration "<Begründung>"`, wenn `kb/` unangetastet bleibt. Beide Marker landen als
eigene Zeile im `CHANGES.md`-Eintrag (`**Breaking Change:**` vor `**Migration:**`) und werden
von `docs verify` unabhängig
voneinander geprüft[^s-version-part-nomenclature-and-breaking-change-gate-session-2026-09-02].
Siehe [[KB Stack Versioning]]. `check` ist der einzige
Befehl, der einen Netzaufruf machen darf - ohne Schlüssel, mit Timeout und injizierbarem Befehl, der einen Netzaufruf machen darf - ohne Schlüssel, mit Timeout und injizierbarem
Fetch, damit Tests nie ein Netz Fetch, damit Tests nie ein Netz
berühren[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30] berühren[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30]
@@ -147,6 +158,50 @@ ist[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30].
## Historie ## Historie
- 2026-09-02 - `2.5.0` (Commit `31662dc`, 806 Tests grün, 6 neu): Gitea-Issue #26 geschlossen.
`instructions/dev/version-parts.md` (neu, `instructions/dev/` - kein Verweis aus einem
ausgelieferten Artefakt, `instructions verify` hätte einen dangelnden Verweis nach
`dist export` gemeldet) trennt die Kompatibilitäts- von der Migrationsfrage: Drop-in-Test in
beiden Richtungen, Katalog der Brüche mit unangetastetem `kb/` (Update-Pfad, Artefaktname,
Import-Name, Flag/Envvar, Shape einer maschinengelesenen Datei), `2.0.0` als Fallbeispiel.
`version bump` bekommt `--breaking "<was aufhört zu funktionieren>"`, bei jedem
Grenzübertritt Pflicht und auf jedem anderen Bump verweigert; `docs verify` prüft das über
eine zweite, von der Migrationsprüfung unabhängige Regel. `stack-dev` bekommt einen
Entscheidungspunkt: kein Grenzübertritt aus eigener Initiative, erst Bruch, Handarbeit je
Instanz und Alternativen (Shim, aufschieben/bündeln, kompatibel/brechend mit
Deprecation-Fenster aufspalten) vorlegen, dann
Freigabe[^s-version-part-nomenclature-and-breaking-change-gate-session-2026-09-02].
- 2026-09-02 - `2.2.3`-`2.4.1`: die vierstufige Sequenz aus Issue #36 (Publish-Remote-Gate
scharf, Lesepfad gehärtet, Root-Auflösung und Bibliotheksgrenze, [[MCP-Leseserver]]) plus
Menschendoku. `.wikitool-remotes.json` war trotz gegenteiliger Dokumentation nicht gesetzt -
angelegt und gegen ein erfundenes Ziel gegengeprüft (Exit 42). Sechs Lesepfad-Härtungen:
gelöschter ReDoS-Zweig in der Ranking-Funktion, ein 30-Sekunden-`rg`-Timeout, ein
YAML-Alias-Refusal (267 B → 672.603 Knoten reproduziert), `CSafeLoader` (265 ms → 54 ms),
gemeldetes statt geschlucktes kaputtes Frontmatter, ein Korpus-Cache, der nie einen
schmutzigen Arbeitsbaum cacht. `config.ROOT` und alle abgeleiteten Pfade waren zur Importzeit
gebunden - jetzt lazy aufgelöst (`$CHEMENU_ROOT` → Walk-up), mit einem eigenen
`config.reset()` gegen eine Falle, in die `monkeypatch`s eigener Teardown sonst zurückläuft.
Neuer typerloser Lesekern (`search/service.py`, `lint_core.py`, `types_core.py`) hinter
`chemenu.api.Corpus`, darüber der MCP-Server: `search`/`types`/`describe_type`/`lint`/`status`,
strukturell ohne Schreibpfad, jede Antwort mit Commit-Stempel. `wikitool search` end-to-end
593 ms → 347 ms. Menschendoku `INSTALL-MCP.md`, ausgeliefert über `dist_cmd.ROOT_FILES`.
Siehe [[Publish-Remote Gate]], [[Mass-Update Gate]],
[[MCP-Leseserver]][^s-mcp-read-server-implementation-session-2026-09-02]
- 2026-09-01 - `2.1.0`-`2.2.2`: Vorbereitung und Durchführung der Veröffentlichung. `dist export`
bekommt `REQUIRED_ROOT_FILES` (fehlende Lizenzdateien lassen den Export scheitern statt still
weiterzulaufen) und `find_leaks()` (strukturelle Prüfung des fertigen Export-Plans gegen
Personalisierungsdateien, `instructions/dev/`, `kb/`- und `raw/`-Inhalte - bewusst kein
Text-Muster-Scan, weil der eigene Hostname legitim in `INSTALL.md`/`version.py` steht). `publish`
bekommt ein drittes Gate: das **Publish-Remote-Gate** prüft die aufgelöste Push-URL (nicht den
Remote-Namen) gegen eine optionale, gitignorete `.wikitool-remotes.json` und hat als einziges
der drei Gates keinen `--confirm`-Token - der Weg daran vorbei ist ein bewusster Edit der
Datei durch den Nutzer, nie durch einen Agenten. `doctor` bekommt den `publish-remotes`-Check.
Ein `raw_dir`-Testfixture löste `config.ROOT` gegen das echte Repo-Root statt die Fixture auf
und bestand nur, weil zufällig ein Verzeichnis existierte, das der Korpus-Schnitt entfernte -
in der Fixture geschlossen, nicht im einzelnen Test, aus demselben Grund wie Gitea #8.
Siehe [[Chemenu]], [[Mass-Update Gate]], [[Publish-Remote Gate]],
[[Delete Rather Than Anonymize]],
[[Dual Licensing by File Plan]][^s-public-release-corpus-purge-and-history-squash-session-2026-09-01][^s-publish-remote-gate-and-issue-triage-session-2026-09-01]
- 2026-09-01 - `2.0.0` (Commit `9a7abe6`, 730 Tests grün): das Python-Paket heißt `chemenu` - 2026-09-01 - `2.0.0` (Commit `9a7abe6`, 730 Tests grün): das Python-Paket heißt `chemenu`
statt `wiki_tools`, **das Kommando bleibt `wikitool`**. Der Import-Name eines Pakets ist ein statt `wiki_tools`, **das Kommando bleibt `wikitool`**. Der Import-Name eines Pakets ist ein
flacher globaler Namensraum ohne Kollisionsschutz, `wiki_tools` war dafür zu generisch; flacher globaler Namensraum ohne Kollisionsschutz, `wiki_tools` war dafür zu generisch;
@@ -249,6 +304,10 @@ ist[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30].
- [[Structural Enforcement over Documented Rule]] - [[Structural Enforcement over Documented Rule]]
- [[Optional Instance Context File]] - [[Optional Instance Context File]]
- [[Source - Conversation - ENVIRONMENT.md as an Optional Third Session-Level File Session 2026-08-31]] - [[Source - Conversation - ENVIRONMENT.md as an Optional Third Session-Level File Session 2026-08-31]]
- [[Source - Public Release, Corpus Purge and History Squash Session 2026-09-01]]
- [[Source - Publish-Remote Gate and Issue Triage Session 2026-09-01]]
- [[Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01]]
- [[MCP-Leseserver]]
## Fußnoten ## Fußnoten
@@ -261,3 +320,7 @@ ist[^s-conversation-versioning-ci-cd-and-content-migration-session-2026-08-30].
[^s-conversation-agents-md-skill-restructuring-session-2026-08-04]: [[Source - Conversation - AGENTS.md Skill Restructuring Session 2026-08-04]] [^s-conversation-agents-md-skill-restructuring-session-2026-08-04]: [[Source - Conversation - AGENTS.md Skill Restructuring Session 2026-08-04]]
[^s-conversation-hardening-the-test-suite-against-silent-environment-dependencies-session-2026-08-31]: [[Source - Conversation - Hardening the Test Suite Against Silent Environment Dependencies Session 2026-08-31]] [^s-conversation-hardening-the-test-suite-against-silent-environment-dependencies-session-2026-08-31]: [[Source - Conversation - Hardening the Test Suite Against Silent Environment Dependencies Session 2026-08-31]]
[^s-llm-improvements-codex-analysis]: [[Source - LLM Improvements Codex Analysis]] [^s-llm-improvements-codex-analysis]: [[Source - LLM Improvements Codex Analysis]]
[^s-public-release-corpus-purge-and-history-squash-session-2026-09-01]: [[Source - Public Release, Corpus Purge and History Squash Session 2026-09-01]]
[^s-publish-remote-gate-and-issue-triage-session-2026-09-01]: [[Source - Publish-Remote Gate and Issue Triage Session 2026-09-01]]
[^s-mcp-read-server-implementation-session-2026-09-02]: [[Source - MCP Read Server Implementation Session 2026-09-02]]
[^s-version-part-nomenclature-and-breaking-change-gate-session-2026-09-02]: [[Source - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02]]
+6 -6
View File
@@ -13,12 +13,12 @@ The page tables live in a generated `INDEX.md` inside each collection, linked be
## Statistics ## Statistics
- **Total Pages:** 170 - **Total Pages:** 180
- **Comparisons:** 1 - **Comparisons:** 1
- **Concepts:** 76 - **Concepts:** 80
- **Entities:** 72 - **Entities:** 72
- **Sources:** 21 - **Sources:** 27
- **Last Updated:** 2026-09-01 - **Last Updated:** 2026-09-02
--- ---
@@ -27,9 +27,9 @@ The page tables live in a generated `INDEX.md` inside each collection, linked be
| Collection | Pages | Index | | Collection | Pages | Index |
|------------|------:|-------| |------------|------:|-------|
| `comparisons/` | 1 | [comparisons/INDEX.md](comparisons/INDEX.md) | | `comparisons/` | 1 | [comparisons/INDEX.md](comparisons/INDEX.md) |
| `concepts/` | 76 | [concepts/INDEX.md](concepts/INDEX.md) | | `concepts/` | 80 | [concepts/INDEX.md](concepts/INDEX.md) |
| `entities/` | 72 | [entities/INDEX.md](entities/INDEX.md) | | `entities/` | 72 | [entities/INDEX.md](entities/INDEX.md) |
| `sources/` | 21 | [sources/INDEX.md](sources/INDEX.md) | | `sources/` | 27 | [sources/INDEX.md](sources/INDEX.md) |
### entities/ ### entities/
+48
View File
@@ -49,3 +49,51 @@ gesetzt. Dieser Eintrag ist der erste des neuen Logs.
und `doctor` grün. und `doctor` grün.
--- ---
## [2026-09-01] update | Chemenu-Projektseite auf den oeffentlichen Stand gebracht
Die Seite beschrieb sich als persoenliches IT-Wissenswiki; seit der Veroeffentlichung ist diese Instanz Testbett und oeffentliche Demo. Beschreibung, Zweck und Lizenz nachgezogen, die historische Aussage ueber die monolithische AGENTS.md als Ausgangspunkt datiert statt geloescht.
---
## [2026-09-01] ingest | raw/notes/Conversation Transcript - Public Release, Corpus Purge and History Squash Session 2026-09-01.md
Source-Seite angelegt, Chemenu und wikitool aktualisiert, zwei neue Concept-Seiten (Delete Rather Than Anonymize, Dual Licensing by File Plan), quer verlinkt.
---
## [2026-09-01] ingest | raw/notes/Conversation Transcript - Publish-Remote Gate and Issue Triage Session 2026-09-01.md
Source-Seite angelegt, neue Concept-Seite Publish-Remote Gate, wikitool-Historie ergaenzt, quer verlinkt mit Mass-Update Gate und Chemenu.
---
## [2026-09-01] ingest | raw/notes/Conversation Transcript - Private-Instance Merge Correction and Issue 30 Session 2026-09-01.md
Source-Seite angelegt, Publish-Remote-Gate-Seite um die gemessene Merge-Semantik erweitert, quer verlinkt.
---
## [2026-09-02] ingest | MCP Read Server Implementation Session 2026-09-02
Source-Seite und neues Concept MCP-Leseserver angelegt; wikitool, Chemenu, Publish-Remote Gate und Iteration and Cost Limits aktualisiert. Deckt Issue #36s Sequenz (#34/#33/#31/#19), INSTALL-MCP.md und Issue #37 ab.
---
## [2026-09-02] update | qmd - Sprachangabe korrigiert
Direkt gegen tobi/qmd auf GitHub geprueft: TypeScript statt der geratenen 'Go oder Rust'-Angabe. Neue Quelle Source - qmd - GitHub Repository (raw/documents/), confidence_base 0.85 -> 0.80 neu begruendet, Installationsbeispiel korrigiert.
---
## [2026-09-02] update | MCP Read Server Implementation Session 2026-09-02 - Korrektur
Aufgaben-Checkbox fuer die qmd.md-Korrektur nachgezogen (war [ ], ist erledigt) und ein Korrektur-Hinweis ergaenzt: der Fidelity-Block des Rohtranskripts kuendigte ein zweites Transkript fuer die qmd-Korrektur an, das nie geschrieben wurde - die Korrektur lief stattdessen als eigene Quellen-Verifikation. raw/ ist unveraenderlich, die Korrektur steht deshalb auf der Source-Seite.
---
## [2026-09-02] ingest | raw/notes/Conversation Transcript - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02.md
Neue Source-Seite, Korrektur der veralteten MAJOR-als-Migration-Formulierung in wikitool.md und KB Stack Versioning.md, Nachtrag in Chemenu.md (Issue #26 geschlossen).
---
+27 -2
View File
@@ -8,8 +8,8 @@ inline `[^cite-id]` footnote).
## Coverage Summary ## Coverage Summary
- **Total raw files:** 21 - **Total raw files:** 26
- **Covered:** 21 - **Covered:** 26
- **Uncovered:** 0 - **Uncovered:** 0
--- ---
@@ -31,6 +31,11 @@ inline `[^cite-id]` footnote).
- Covered by: [[Source - LLM Wiki Pattern]] - Covered by: [[Source - LLM Wiki Pattern]]
- Cited by: [[Andrej Karpathy]], [[ChatGPT]], [[Claude Code]], [[Dataview]], [[Knowledge Compounding]], [[LLM Wiki Pattern]], [[Marp]], [[Memex]], [[NotebookLM]], [[Obsidian]], [[Obsidian Web Clipper]], [[OpenAI Codex]], [[OpenCode]], [[Pi]], [[RAG]], [[Three-Layer Architecture]], [[Tolkien Gateway]], [[Vannevar Bush]], [[qmd]] - Cited by: [[Andrej Karpathy]], [[ChatGPT]], [[Claude Code]], [[Dataview]], [[Knowledge Compounding]], [[LLM Wiki Pattern]], [[Marp]], [[Memex]], [[NotebookLM]], [[Obsidian]], [[Obsidian Web Clipper]], [[OpenAI Codex]], [[OpenCode]], [[Pi]], [[RAG]], [[Three-Layer Architecture]], [[Tolkien Gateway]], [[Vannevar Bush]], [[qmd]]
### `raw/documents/qmd - GitHub Repository.md`
- Covered by: [[Source - qmd - GitHub Repository]]
- Cited by: [[qmd]]
### `raw/notes/Arch Linux Cheat Sheet.md` ### `raw/notes/Arch Linux Cheat Sheet.md`
- Covered by: [[Source - Arch Linux Cheat Sheet]] - Covered by: [[Source - Arch Linux Cheat Sheet]]
@@ -71,11 +76,31 @@ inline `[^cite-id]` footnote).
- Covered by: [[Source - Conversation - Issue Triage Labels and TODO Retirement Session 2026-08-31]] - Covered by: [[Source - Conversation - Issue Triage Labels and TODO Retirement Session 2026-08-31]]
- Cited by: [[Chemenu]], [[Detect-Repair Asymmetry]], [[Gitea]], [[Gitea Actions]], [[Gitea MCP Server]], [[Issue Label Scheme]], [[KB Stack Versioning]], [[wikitool]] - Cited by: [[Chemenu]], [[Detect-Repair Asymmetry]], [[Gitea]], [[Gitea Actions]], [[Gitea MCP Server]], [[Issue Label Scheme]], [[KB Stack Versioning]], [[wikitool]]
### `raw/notes/Conversation Transcript - MCP Read Server Implementation Session 2026-09-02.md`
- Covered by: [[Source - MCP Read Server Implementation Session 2026-09-02]]
- Cited by: [[Chemenu]], [[Iteration and Cost Limits]], [[MCP-Leseserver]], [[Publish-Remote Gate]], [[wikitool]]
### `raw/notes/Conversation Transcript - Nightly Drift-Check Workflow and doctor's Bootstrap Gap Session 2026-08-31.md` ### `raw/notes/Conversation Transcript - Nightly Drift-Check Workflow and doctor's Bootstrap Gap Session 2026-08-31.md`
- Covered by: [[Source - Conversation - Nightly Drift-Check Workflow and doctor's Bootstrap Gap Session 2026-08-31]] - Covered by: [[Source - Conversation - Nightly Drift-Check Workflow and doctor's Bootstrap Gap Session 2026-08-31]]
- Cited by: [[CI Integration]], [[Gitea Actions]] - Cited by: [[CI Integration]], [[Gitea Actions]]
### `raw/notes/Conversation Transcript - Private-Instance Merge Correction and Issue 30 Session 2026-09-01.md`
- Covered by: [[Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01]]
- Cited by: [[Chemenu]], [[Delete Rather Than Anonymize]], [[Publish-Remote Gate]], [[wikitool]]
### `raw/notes/Conversation Transcript - Public Release, Corpus Purge and History Squash Session 2026-09-01.md`
- Covered by: [[Source - Public Release, Corpus Purge and History Squash Session 2026-09-01]]
- Cited by: [[Chemenu]], [[Delete Rather Than Anonymize]], [[Dual Licensing by File Plan]], [[wikitool]]
### `raw/notes/Conversation Transcript - Publish-Remote Gate and Issue Triage Session 2026-09-01.md`
- Covered by: [[Source - Publish-Remote Gate and Issue Triage Session 2026-09-01]]
- Cited by: [[Chemenu]], [[Publish-Remote Gate]], [[wikitool]]
### `raw/notes/Conversation Transcript - Two Round-Trip Defects Found by an Ingest Session 2026-08-31.md` ### `raw/notes/Conversation Transcript - Two Round-Trip Defects Found by an Ingest Session 2026-08-31.md`
- Covered by: [[Source - Conversation - Two Round-Trip Defects Found by an Ingest Session 2026-08-31]] - Covered by: [[Source - Conversation - Two Round-Trip Defects Found by an Ingest Session 2026-08-31]]
+26 -2
View File
@@ -1,3 +1,10 @@
---
profile: sources
outbound:
any: [is-evidence-for, defined-in, see-also]
required_by_stack: true
---
# kb/sources/ - Collection Contract # kb/sources/ - Collection Contract
One page per ingested source. A source page is the bridge between the untrusted material in One page per ingested source. A source page is the bridge between the untrusted material in
@@ -9,8 +16,14 @@ concluded from it. Where the source is wrong, say what it claims and let the sub
carry the correction. A source page that has been improved beyond its source is no longer carry the correction. A source page that has been improved beyond its source is no longer
evidence for anything. evidence for anything.
Inherits [kb/CONTRACT.md](../CONTRACT.md) - naming, tone, linking, provenance and confidence Inherits [kb/CONTRACT.md](../CONTRACT.md) for the rules the stack enforces - linking mechanics,
are defined there and are not restated here. provenance, citation, the confidence machinery - and
[kb/CONVENTIONS.md](../CONVENTIONS.md) for what this instance decided: language, naming forms,
tone, relationship labels, the confidence rubric. Neither is restated here.
**This collection is `required_by_stack`.** `sources coverage`, `[^cite-id]` resolution and
`kb/provenance.md` resolve against it by name, so unlike every other collection it may not be
renamed or dropped - its authoring rules below are the instance's, its existence is not.
## Types offered ## Types offered
@@ -27,6 +40,17 @@ The `raw_files:`/`source_url:`/citation rules are shared and live in
- `tools/wikitool sources trace --raw <path>` answers "what did we learn from this?"; - `tools/wikitool sources trace --raw <path>` answers "what did we learn from this?";
`tools/wikitool sources coverage` lists raw files no source page claims yet. `tools/wikitool sources coverage` lists raw files no source page claims yet.
## Authorised labels
The `outbound:` block above is what `wikitool lint` and `xref add` check: which labels a page in
this collection may use, per destination. The catalogue they are drawn from - and what each one
asserts - is [instructions/link-taxonomy.md](../../instructions/link-taxonomy.md), which binds
nothing on its own.
Deliberately narrow. A source page is evidence *about* a source; almost everything it would want to say is already carried by `raw_files:`, `sources:` and `[^cite-id]`, which are the mechanical provenance path rather than authored edges.
Adding a label here is a deliberate contract change, not a way around a refusal.
## Outbound linking ## Outbound linking
A source page links to every entity and concept it produced or updated. A source page links to every entity and concept it produced or updated.
+7 -1
View File
@@ -2,7 +2,7 @@
# kb/sources/ - Index # kb/sources/ - Index
21 page(s). Regenerated by `wikitool index rebuild`. 27 page(s). Regenerated by `wikitool index rebuild`.
## All ## All
@@ -28,5 +28,11 @@
| [[Source - LLM Improvements Sonnet Analysis]] | notes | Sonnet-Analyse, die AGENTS.md und wikitool mit Farzas Gist und awesome-llm-wiki vergleicht und die Codex-Analyse um konkrete Empfehlungen zu Qualitätsschwellen, Stilrichtlinie, Auditrhythmus und Skalierung ergänzt | 2026-08-03 | | [[Source - LLM Improvements Sonnet Analysis]] | notes | Sonnet-Analyse, die AGENTS.md und wikitool mit Farzas Gist und awesome-llm-wiki vergleicht und die Codex-Analyse um konkrete Empfehlungen zu Qualitätsschwellen, Stilrichtlinie, Auditrhythmus und Skalierung ergänzt | 2026-08-03 |
| [[Source - LLM Wiki Pattern]] | article | Grundlegendes Muster für persönliche Wissensbasen mit LLMs: ein dauerhaftes Wiki schrittweise pflegen, statt es aus den Quellen neu herzuleiten. | 2026-07-26 | | [[Source - LLM Wiki Pattern]] | article | Grundlegendes Muster für persönliche Wissensbasen mit LLMs: ein dauerhaftes Wiki schrittweise pflegen, statt es aus den Quellen neu herzuleiten. | 2026-07-26 |
| [[Source - LLM Wiki v2]] | article | Erweitertes LLM-Wiki-Muster mit Praxiserfahrungen aus agentmemory zu Memory Lifecycle, Confidence Scoring, Wissensgraph und Automatisierung. | 2026-07-26 | | [[Source - LLM Wiki v2]] | article | Erweitertes LLM-Wiki-Muster mit Praxiserfahrungen aus agentmemory zu Memory Lifecycle, Confidence Scoring, Wissensgraph und Automatisierung. | 2026-07-26 |
| [[Source - MCP Read Server Implementation Session 2026-09-02]] | notes | Sitzung, die die Sequenz aus Issue #36 umsetzt: Publish-Remote Gate scharf, Lesepfad gehaertet, Root-Aufloesung und Bibliotheksgrenze gezogen, MCP-Leseserver gebaut - vier Versionsstufen 2.2.3 bis 2.4.0, dazu INSTALL-MCP.md und Issue #37. | 2026-09-02 |
| [[Source - Private-Instance Merge Correction and Issue 30 Session 2026-09-01]] | notes | Sitzung, die eine ungeprueft niedergeschriebene Merge-Behauptung in private-instance.md durch einen empirischen Test widerlegt, die Prozedur korrigiert (2.2.1) und Issue #30 mit einem getesteten Skript sowie zwei Architekturvorschlaegen anlegt. | 2026-09-01 |
| [[Source - Public Release, Corpus Purge and History Squash Session 2026-09-01]] | notes | Sitzung, die den Chemenu-Stack von einer privaten Testinstanz in ein oeffentliches Repo ueberfuehrt: Korpus geloescht statt anonymisiert, Git-History auf einen Commit gesquashed, AGPL-3.0/CC-BY-4.0-Dual-Lizenz gewaehlt, dist export um einen Leak-Canary gehaertet. | 2026-09-01 |
| [[Source - Publish-Remote Gate and Issue Triage Session 2026-09-01]] | notes | Sitzung, die ein drittes, Token-loses Gate fuer publish baut, instructions/private-instance.md schreibt, sechs Gitea-Issues auf den Rename und die neue Architektur nachzieht und die Actions-Run-Historie entfernen laesst. | 2026-09-01 |
| [[Source - qmd - GitHub Repository]] | document | GitHub-API-Metadaten, package.json und README-Auszuege von tobi/qmd: TypeScript/Node/Bun statt Go oder Rust, BM25 (SQLite FTS5) plus Vektor-Suche (sqlite-vec) plus LLM-Reranking ueber node-llama-cpp. | 2026-09-02 |
| [[Source - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02]] | notes | Sitzung, die die Versionsstelle als Kompatibilitaets- statt Migrationsfrage praezisiert und einen Freigabe-Ablauf fuer Breaking Changes in stack-dev einfuehrt | 2026-09-02 |
| [[Source - Wine]] | notes | Wine-Konfiguration für Arch Linux: pacman-NoExtract-Einstellungen und Bottles-Runtime-Optionen einschließlich Proton- und Lutris-Varianten. | 2026-08-01 | | [[Source - Wine]] | notes | Wine-Konfiguration für Arch Linux: pacman-NoExtract-Einstellungen und Bottles-Runtime-Optionen einschließlich Proton- und Lutris-Varianten. | 2026-08-01 |
@@ -0,0 +1,109 @@
---
type: types/source.md
source_type: notes
author: Torben
raw_files: [raw/notes/Conversation Transcript - MCP Read Server Implementation Session 2026-09-02.md]
source_language: de
date: 2026-09-02
tags: []
entities: [wikitool, Chemenu]
concepts: [Publish-Remote Gate, Mass-Update Gate, Iteration and Cost Limits, MCP-Leseserver]
summary: 'Sitzung, die die Sequenz aus Issue #36 umsetzt: Publish-Remote Gate scharf, Lesepfad gehaertet, Root-Aufloesung und Bibliotheksgrenze gezogen, MCP-Leseserver gebaut - vier Versionsstufen 2.2.3 bis 2.4.0, dazu INSTALL-MCP.md und Issue #37.'
---
# Source: MCP Read Server Implementation Session 2026-09-02
**Autor:** Torben
**Datum:** 2026-09-02
**Raw-Dateien:** raw/notes/Conversation Transcript - MCP Read Server Implementation Session 2026-09-02.md
**Typ:** Notes
## Zusammenfassung
Diese Sitzung arbeitet die vierstufige Sequenz aus Issue #36 ab, dem Sammel-Issue für den Weg
zum MCP-Leseserver: #34 (Publish-Remote-Gate scharf stellen), #33 (Lesepfad vor der Exposition
härten), #31 (Root-Auflösung von der Importzeit lösen und eine Bibliotheksgrenze ziehen), #19
(der Leseserver selbst). Jeder Schritt endet mit einem Versions-Bump und einem Testlauf; die
Reihenfolge folgt dem Master-Issue, weil #33 dieselben Dateien anfasst, die #31 strukturell
umbaut, und #31 die Grenze liefert, auf der #19 aufsetzt.
`.wikitool-remotes.json` fehlte in diesem Checkout trotz Dokumentation, die das Gegenteil
behauptete - angelegt und gegen ein erfundenes Ziel gegengeprüft (Exit 42). Der Lesepfad bekam
sechs Fixes gegen einen 267-Byte-YAML-Alias, der zu 672.603 Knoten expandiert, gegen einen
ReDoS-Zweig in der Ranking-Funktion, einen fehlenden Subprozess-Timeout, und einen Korpus-Cache,
der nie einen schmutzigen Arbeitsbaum cacht. `config.ROOT` und alle abgeleiteten Pfade waren zur
Importzeit gebunden; die Auflösung ist jetzt lazy (`CHEMENU_ROOT` → Walk-up), und der reine
Lesekern (`search/service.py`, `lint_core.py`, `types_core.py`) importiert kein `typer` mehr.
Der MCP-Server (`tools/chemenu/mcp/`) exponiert `search`/`types`/`describe_type`/`lint`/`status`
über `chemenu.api.Corpus` - strukturell ohne Schreibpfad, mit Commit-Stempel auf jeder Antwort
und einer Startverweigerung, falls Telemetrie in den bedienten Baum schreiben würde.
Nach Freigabe des 37-Datei-Changesets (Mass-Update-Gate, Token `46442f4419c1`) folgten
`INSTALL-MCP.md` für Menschen, ein Verweis auf die separate Traefik-ForwardAuth-Middleware
(`gitea-mcp-forward-auth`), und Issue #37 für das noch fehlende Container-Image - mit den
konkreten CI-Vorlagen aus `gitea-mcp-forward-auth` (Registry-Push) und `gitea-mcp`
(Dockerfile-Form, aber DockerHub statt der eigenen Registry). Alle vier Sequenz-Issues wurden
geschlossen, #36 blieb offen, weil sein eigenes Abschlusskriterium - ein Konsument, der
nachweislich über die Middleware antwortet - erst mit #37 erfüllbar ist.
## Kernaussagen
- Gemessen: Korpus-Parse 265 ms → 54 ms (`CSafeLoader`), `wikitool search` end-to-end
593 ms → 347 ms; die verbleibenden ~262 ms sind Modulimport und entfallen erst im residenten
MCP-Prozess.
- Der ReDoS-Zweig (`_contains` mit `re.search` gegen nutzergesteuerten Regex) wurde gelöscht,
nicht begrenzt - `rg` wendet das Muster ohnehin mit einer linearen Engine an, bevor die
Funktion je läuft.
- `monkeypatch.setattr(config, "ROOT", ...)` baute nach der lazy-Auflösung die stale Bindung
beim Teardown wieder auf, weil es den *aufgelösten* alten Wert zurückschreibt - `config.reset()`
musste dazukommen, in derselben autouse-Fixture, die das Problem eine Ebene höher (Umgebungsvariablen)
bereits kannte.
- Der MCP-Server hat keinen Schreibpfad, weil `chemenu.api` nichts unter `chemenu.commands`
importiert - nicht, weil eine Liste gefiltert wird. Ein Test importiert das Servermodul in
einem frischen Interpreter und prüft `sys.modules`.
- Ein Stempel-Bug wurde beim Schreiben des Golden-Tests selbst gefunden: `_stamp()` fragte nach
der *aktuellen* statt der beim Laden tatsächlich gelesenen Revision und hätte bei einem
minimal verzögerten zweiten Zugriff `commit: null` auf einem sauberen Baum liefern können.
- `gitea-mcp` ist als Registry-Vorlage ungeeignet - sein Release-Workflow pusht nach DockerHub
(Fork des Upstream), nicht in die eigene Gitea-Registry.
## Aufgaben
- [x] #34, #33, #31, #19 umgesetzt und mit Abschlusskommentar geschlossen
- [x] `INSTALL-MCP.md` geschrieben, in `INSTALL.md`/`README.md` verlinkt, in `dist export` aufgenommen
- [x] Issue #37 (Container-Image) angelegt, mit neun offenen Entscheidungen benannt
- [ ] #37 selbst umsetzen
- [ ] #36 schließen, sobald #37 den Middleware-Nachweis liefert
- [ ] #23 (Env-Var-Erzwingung) - `CHEMENU_ROOT` wurde von Hand in `_WIKITOOL_ENV` eingetragen
- [x] `kb/entities/tools/qmd.md` - falsche Sprachangabe korrigiert (siehe Korrektur unten)
## Korrektur zum Transkript-Kopf
Der Fidelity-Block des Rohtranskripts sagt: "One of two transcripts cut from this session; the
other covers fixing `kb/entities/tools/qmd.md`". Dieses zweite Transkript wurde nie geschrieben
- `raw/` ist unveränderlich, die Korrektur gehört hierher, nicht in die Datei selbst. Tatsächlich
lief die Korrektur ohne eigenes Transkript: direkt gegen `tobi/qmd` auf GitHub geprüft und als
eigene Quelle mit eigenem Raw-Beleg abgelegt (`Source - qmd - GitHub Repository`,
`raw/documents/qmd - GitHub Repository.md`) - eine Quellen-Verifikation statt eines
Gesprächsprotokolls, was für eine Sprachangaben-Korrektur die passendere Belegform ist.
## Nicht übernommen
- Der vollständige Wortlaut der geprüften Docstrings, Kommentare und Testfälle - das Transkript
benennt Dateien und die tragenden Eigenschaften, der Code selbst ist die Quelle.
- Die exakten neun offenen Entscheidungspunkte aus Issue #37 (Korpus im Image vs. Volume,
Basis-Image, Healthcheck etc.) - dort bereits vollständig dokumentiert, hier nicht dupliziert.
- Der Wortlaut der abgerufenen READMEs von `gitea-mcp-forward-auth` und `gitea-mcp` - nur die
für die Entscheidung relevanten Fakten (Config-Variablen, Workflow-Form, Registry-Ziel)
wurden übernommen.
## Verwandte Entities
- [[wikitool]]
- [[Chemenu]]
## Verwandte Concepts
- [[Publish-Remote Gate]]
- [[Mass-Update Gate]]
- [[Iteration and Cost Limits]]
- [[MCP-Leseserver]]
@@ -0,0 +1,83 @@
---
type: types/source.md
source_type: notes
author: Torben
raw_files: [raw/notes/Conversation Transcript - Private-Instance Merge Correction and Issue 30 Session 2026-09-01.md]
source_language: de
date: 2026-09-01
tags: []
entities: [Chemenu, wikitool]
concepts: [Publish-Remote Gate, Delete Rather Than Anonymize]
summary: 'Sitzung, die eine ungeprueft niedergeschriebene Merge-Behauptung in private-instance.md durch einen empirischen Test widerlegt, die Prozedur korrigiert (2.2.1) und Issue #30 mit einem getesteten Skript sowie zwei Architekturvorschlaegen anlegt.'
---
# Source: Private-Instance Merge Correction and Issue 30 Session 2026-09-01
**Autor:** Torben
**Datum:** 2026-09-01
**Raw-Dateien:** raw/notes/Conversation Transcript - Private-Instance Merge Correction and Issue 30 Session 2026-09-01.md
**Typ:** Notes
## Zusammenfassung
Torben fragte, was in der privaten Instanz nach dem beschriebenen Schema passiert, wenn Upstream
den Demo-Korpus ändert - eine Frage, die eine am selben Tag geschriebene, aber nie getestete
Behauptung in `instructions/private-instance.md` traf. Statt die bestehende Textstelle zu
verteidigen, wurde sie an einem Wegwerf-Repo-Paar empirisch geprüft: Ein `git merge
upstream/main` löst eine geänderte, gelöschte Demo-Seite **nicht** still auf, sondern erzeugt
einen `modify/delete`-Konflikt und lässt die Upstream-Fassung im Arbeitsbaum liegen; eine neu
angelegte Demo-Seite wird dagegen **stillschweigend** gestaged, ohne Konflikt und ohne Meldung.
Ein zweiter, naheliegender Fix (`.gitattributes` mit `merge=ours` für die Inhaltsverzeichnisse)
wurde ebenfalls getestet und ebenfalls widerlegt - der Treiber wirkt nur bei Inhaltskonflikten
auf beidseitig vorhandenen Dateien, nicht bei modify/delete oder Neuanlage.
Die Instruktion wurde korrigiert (Version 2.2.1): Der Merge wird mit `--no-commit` offengehalten,
die Inhaltsverzeichnisse werden auf den Stand vor dem Merge zurückgezwungen, solange `HEAD` noch
dorthin zeigt, erst dann wird committet - gefolgt von einer Kontrolle
(`git diff --name-only $BEFORE HEAD -- kb raw` muss leer sein), die nicht stillschweigend
übersprungen werden kann. Ein eigenständiges, getestetes Skript wurde daraus abgeleitet und in
Issue #30 hinterlegt, zusammen mit zwei Architekturvorschlägen: das Verfahren als `wikitool`-
Kommando statt als Shell-Rezept, oder - als eigentliche Ursachenbehebung - den Demo-Korpus
grundsätzlich von dem Branch fernzuhalten, von dem private Instanzen ihre Maschinerie ziehen.
Anschließend wurde Issue #3 (Chemenu-Rebranding) gegen seine eigenen Abnahmekriterien geprüft
und für sauber befunden, und eine Verdrahtungslücke geschlossen (`tools/CONTRACT.md` kannte das
Publish-Remote-Gate nicht, `gates.md` verlinkte nicht auf die Prozedur, die Chemenu-Projektseite
beschrieb sich noch als privates Wiki) - veröffentlicht als 2.2.2.
## Kernaussagen
- Eine Behauptung über Git-Merge-Verhalten ist erst nach einem Test eine Tatsache. Am selben Tag
geschrieben zu haben ist kein Beleg für Richtigkeit.
- `git merge` behandelt "eine Datei geändert" und "eine Datei neu angelegt" unterschiedlich: Nur
Ersteres erzeugt einen sichtbaren Konflikt. Eine Prozedur, die nur den Konfliktfall bedenkt,
übersieht die stille Neuanlage.
- `.gitattributes`-Merge-Treiber wie `merge=ours` wirken nur auf Inhaltskonflikte zwischen
beidseitig vorhandenen Dateiversionen, nicht auf modify/delete-Paare oder Neuanlagen.
- Eine Kontrollprüfung, die ein Mensch lesen und verstehen muss, um einen Fehler zu bemerken, ist
schwächer als eine, die bei einem Fehler selbst nicht-null zurückgibt.
- Wiederkehrende Symptome bei jeder Downstream-Instanz sind oft ein Zeichen, dass die eigentliche
Ursache stromaufwärts liegt und dort einmalig behoben werden sollte.
## Aufgaben
- [x] `private-instance.md` korrigiert und als 2.2.1 veröffentlicht
- [x] Issue #30 angelegt (Skript + zwei Architekturvorschläge, hängt an #28)
- [x] Issue #3 gegen Abnahmekriterien geprüft, sauber
- [ ] Issue #30 selbst ist offen - siehe dort für den Entscheidungsstand
## Nicht übernommen
- Die im Transkript vollständig gezeigten Testskript-Läufe (Shell-Ausgabe der Wegwerf-Repos)
sind hier nicht wiederholt - die Kernaussage (welches Verhalten gemessen wurde) ist auf die
Concept-Seite [[Publish-Remote Gate]] und in Issue #30 übernommen, der Beleg bleibt im
Transkript.
## Verwandte Entities
- [[Chemenu]]
- [[wikitool]]
## Verwandte Concepts
- [[Publish-Remote Gate]]
- [[Delete Rather Than Anonymize]]
@@ -0,0 +1,87 @@
---
type: types/source.md
source_type: notes
author: Torben
raw_files: ['raw/notes/Conversation Transcript - Public Release, Corpus Purge and History Squash Session 2026-09-01.md']
source_language: de
date: 2026-09-01
tags: []
entities: [Chemenu, wikitool]
concepts: [Mass-Update Gate, Delete Rather Than Anonymize, Dual Licensing by File Plan]
summary: 'Sitzung, die den Chemenu-Stack von einer privaten Testinstanz in ein oeffentliches Repo ueberfuehrt: Korpus geloescht statt anonymisiert, Git-History auf einen Commit gesquashed, AGPL-3.0/CC-BY-4.0-Dual-Lizenz gewaehlt, dist export um einen Leak-Canary gehaertet.'
---
# Source: Public Release, Corpus Purge and History Squash Session 2026-09-01
**Autor:** Torben
**Datum:** 2026-09-01
**Raw-Dateien:** raw/notes/Conversation Transcript - Public Release, Corpus Purge and History Squash Session 2026-09-01.md
**Typ:** Notes
## Zusammenfassung
Torben bat darum, den Chemenu-Korpus zu bereinigen und das Repo zu veröffentlichen, mit
ausdrücklichem Wunsch nach Teufels-Advokat-Modus und einer kleinen Multi-Agent-Debatte. Drei
parallele Explore-Agenten inventarisierten private Daten, den Distributionsmechanismus und die
Git-History; die History-Prüfung erklärte den Baum fälschlich für „safe to publish", ohne
`USER.md` je zu öffnen — ein Befund, der später den Ausschlag für den vollständigen History-Schnitt
gab. Drei geforkte Debattierer (harte Trennung, geteiltes Upstream, Teufels-Advokat) argumentierten
gegeneinander; die Synthese übernahm „löschen statt anonymisieren" und „History squashen" von der
harten Position, das Clone-mit-Upstream-Modell von der geteilten Position, aber unter der
Bedingung, dass ein Remote-Gate zuerst existiert — eine Bedingung, die der Teufels-Advokat mit
seinem Leck-Argument erzwang.
Nutzer traf vier Entscheidungen über `AskUserQuestion`: Gitea öffentlich schalten (kleinste
Änderung), Korpus chirurgisch löschen, private Instanz als Clone mit Upstream und Gate zuerst,
und als Lizenz AGPL-3.0 (Stack) + CC-BY-4.0 (Inhalte) — die Affero-Variante bewusst wegen Issue
#19 (MCP-Frontend als Netzdienst).
Ausgeführt wurde: Lizenzdateien (AGPL-Text von gnu.org geholt, nicht aus dem Gedächtnis
rekonstruiert), ein Leak-Canary in `dist export` (`find_leaks()`, strukturell statt textbasiert,
weil ein Muster-Scan den eigenen legitimen Host mit ausschließen müsste), die Korpus-Löschung
(108 Seiten statt der im Plan geschätzten ~35, weil 40 Seiten mit generischen Titeln tatsächlich
um die private Infrastruktur herum geschrieben waren), der History-Squash auf einen Commit, und
die Veröffentlichung selbst.
Zwei Annahmen wurden durch Messung widerlegt und korrigiert: Ein Force-Push allein reicht nicht —
der alte HEAD blieb per SHA abrufbar, bis Reflogs auf dem Server verfielen und `gc --prune=now`
lief. Und `rg`-Scans ohne `--hidden` übersehen `.gitea/`, `.github/`, `.vibe/` — ein zweiter
Fund (private Referenzen in `ci.yml`) kam erst über `git grep` zum Vorschein.
## Kernaussagen
- Ein Seitentitel ist der einzige Identifier des Wikis; Löschen (`wikitool rm`) ist dafür billiger
und sicherer als Anonymisieren, das die volle `page-lifecycle`-Prozedur pro Seite verlangt.
- Der eigentliche Preisgeber bei einem Infrastruktur-Handbuch ist die Topologie, nicht der
Hostname — gefälschte IPs entschärfen keine Angriffskarte.
- Ein Force-Push macht alte Commits unreferenziert, aber nicht unerreichbar: Sie bleiben per SHA
fetchbar, bis Server-Reflogs verfallen sind und `git gc --prune=now` gelaufen ist.
- `dist export`s Lizenz-Dateien mussten zur Pflicht werden (`REQUIRED_ROOT_FILES`), weil das
übliche `if source.is_file()`-Muster eine fehlende Lizenz still überspringen würde — bei AGPL
eine Verletzung, sobald eine Instanz öffentlich landet.
- Ein Text-Muster-Scan für Leaks scheitert an legitimen Vorkommen des eigenen Hostnamens; ein
struktureller Scan (welche Pfade/Dateien dürfen nie im Plan stehen) umgeht das.
## Aufgaben
- [x] Korpus bereinigt, History gesquasht, Lizenzen gesetzt, Repo veröffentlicht (in dieser
Sitzung erledigt)
- [ ] Siehe Issue #27 (Decay-/Lint-Ausschluss für mitgelieferte Seiten) und #28 (Demo-Korpus als
eigene Fixture) für Folgearbeit aus dieser Sitzung
## Nicht übernommen
- Die konkreten Namen und Details der gelöschten privaten Infrastruktur (Hostnamen, IP-Bereiche,
persönliche Angaben) sind bewusst nicht in diese Source-Seite übernommen — sie zu wiederholen
widerspräche dem Zweck der Sitzung. Wo sie als Beispiel dienen mussten, steht hier nur die Art
der Information (z. B. "eine Homelab-Cluster-Dokumentation"), nie der Wortlaut.
- Der volle Wortlaut der drei Debattenpositionen ist nicht übernommen - nur ihre tragenden
Argumente und was aus ihnen in die Synthese einging. Der vollständige Text steht im Transkript.
## Verwandte Entities
- [[Chemenu]]
- [[wikitool]]
## Verwandte Concepts
- [[Mass-Update Gate]]
@@ -0,0 +1,71 @@
---
type: types/source.md
source_type: notes
author: Torben
raw_files: [raw/notes/Conversation Transcript - Publish-Remote Gate and Issue Triage Session 2026-09-01.md]
source_language: de
date: 2026-09-01
tags: []
entities: [Chemenu, wikitool]
concepts: [Publish-Remote Gate, Mass-Update Gate, Issue Label Scheme]
summary: Sitzung, die ein drittes, Token-loses Gate fuer publish baut, instructions/private-instance.md schreibt, sechs Gitea-Issues auf den Rename und die neue Architektur nachzieht und die Actions-Run-Historie entfernen laesst.
---
# Source: Publish-Remote Gate and Issue Triage Session 2026-09-01
**Autor:** Torben
**Datum:** 2026-09-01
**Raw-Dateien:** raw/notes/Conversation Transcript - Publish-Remote Gate and Issue Triage Session 2026-09-01.md
**Typ:** Notes
## Zusammenfassung
Direkte Fortsetzung der Veröffentlichungssitzung: Torben bat um eine Remote-Allowlist für
`publish`, eine Bitte, vor dem Öffentlich-Schalten von Gitea zu warten und einen Spickzettel
dafür, und ein Kurz-Howto für die lokale Dev-Umgebung - dazu, nach eigenem Ermessen, Issues zu
pflegen. Gebaut wurde das **Publish-Remote-Gate**: Es prüft die aufgelöste Push-URL (nicht den
Remote-Namen, weil ein umgebogener `origin` sonst durchrutschen würde) gegen eine optionale,
gitignorete Allowlist-Datei und hat als einziges der drei Gates keinen Freigabe-Token - der Weg
daran vorbei ist ein bewusster Edit der Datei durch den Menschen. `instructions/private-instance.md`
beschreibt seither das Clone-mit-Upstream-Setup, mit dem Gate als Schritt vor dem ersten `publish`.
Sechs offene Issues wurden auf den Chemenu-Rename hin durchgesehen; mehrere trugen noch den
alten Paketpfad. Drei neue Issues entstanden aus Punkten, die im Tagesverlauf entschieden und
dann zurückgestellt worden waren (Handbuch-Vorbedingung, Demo-vs-Testbett-Konflikt, veraltete
Issue-Texte). Nach dem Öffentlich-Schalten wurde anonym end-to-end geprüft (Klon, Release-Feed,
Distributionsweg), `INSTALL.md` von "Repo ist privat" auf den öffentlichen Zustand umgestellt,
und auf Bitte des Nutzers die Gitea-Actions-Run-Historie über die REST-API entfernt, nachdem
sich herausstellte, dass weder das MCP-Werkzeug noch eine sichtbare UI-Schaltfläche das können.
## Kernaussagen
- Ein Allowlist-Gate muss die **aufgelöste Push-URL** prüfen, nicht den Remote-Namen - sonst
schützt es nicht vor einem umbenannten oder umgebogenen Remote.
- Ein Gate, dessen Frage eine stehende Eigenschaft des Checkouts ist (nicht ein einzelnes
Changeset), braucht keinen Freigabe-Token - der Mensch löst es durch einen bewussten
Datei-Edit, nie ein Agent durch einen Bypass.
- Ein Werkzeugvertrag (`tools/CONTRACT.md`) muss jeden Fehlerfall eines Kommandos nennen; ein
neuer Exit-42-Pfad, der dort fehlt, ist eine Lücke, die kein automatischer Check findet.
- Ein MCP-Server kann weniger können als die zugrunde liegende API - hier: Actions-Runs
anzeigen/erneut starten, aber nicht löschen, obwohl die REST-API die Route hat.
## Aufgaben
- [x] Publish-Remote-Gate, `private-instance.md`, Issue-Pflege in dieser Sitzung erledigt
- [ ] #4, #5 tragen laut #29 noch veraltete Pfade und sind noch nicht nachgezogen
## Nicht übernommen
- Der vollständige Wortlaut des Gitea-Spickzettels ist nicht auf diese Seite übernommen - er
steht im Transkript und in der Chat-Antwort an den Nutzer, ist aber keine dauerhafte
Wiki-Aussage, sondern eine einmalige Handlungsanweisung.
## Verwandte Entities
- [[Chemenu]]
- [[wikitool]]
## Verwandte Concepts
- [[Publish-Remote Gate]]
- [[Mass-Update Gate]]
- [[Issue Label Scheme]]
@@ -0,0 +1,75 @@
---
type: types/source.md
source_type: notes
author: Torben Nehmer
raw_files: [raw/notes/Conversation Transcript - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02.md]
source_language: de
date: 2026-09-02
tags: []
entities: [wikitool, Chemenu]
concepts: [KB Stack Versioning]
summary: Sitzung, die die Versionsstelle als Kompatibilitaets- statt Migrationsfrage praezisiert und einen Freigabe-Ablauf fuer Breaking Changes in stack-dev einfuehrt
---
# Source: Version Part Nomenclature and Breaking Change Gate Session 2026-09-02
**Autor:** Torben Nehmer
**Datum:** 2026-09-02
**Raw-Dateien:** raw/notes/Conversation Transcript - Version Part Nomenclature and Breaking Change Gate Session 2026-09-02.md
**Typ:** Notes
## Zusammenfassung
Umsetzung von Gitea-Issue #26: Die Doku des Stacks führte für die Wahl der Versionsstelle zwei
Fragen zusammen, die nicht dieselbe sind - ob der Korpus migriert werden muss, und ob der
Wechsel ein Drop-in-Ersatz ist. Nur `version bump --help` unterschied korrekt; die drei
prosaischen Stellen (`stack-dev`, `version.py`-Docstring, `INSTALL.md`) beschrieben MAJOR als
Migrationsfrage. Der `2.0.0`-Rebranding-Bump hatte genau daran zuerst `1.9.0` statt `--major`
angesetzt.
Der Nutzer schärfte die Regel während der Sitzung zu einem konkreten zweiseitigen Test nach:
"die neue version ist kein drop-in replacement. Sobald irgendwie Hand angelegt werden muss, sei
es durch den user oder durch ein Migrationsscript, ist es ein major version change. selbiges
gilt, wenn ein update nicht rückgängig gemacht werden kann [...] in allen Fällen muss bei einem
Major version change ein 'Breaking Change' vermerkt werden. breaking changes sind damit teuer.
passe stack-dev so an, dass in diesen Fällen zwingend der user informiert, Alternativen
aufgezeigt und eine freigabe eingeholt wird." Zwei Auswahlentscheidungen davor: Durchsetzung im
Code statt reiner Prosa (weil Prosa bereits einmal gedriftet war), und die Freigabe als
"Decision point" statt als Gate-Sprache, um die drei echten code-erzwungenen Gates nicht zu
verwässern.
## Kernaussagen
- Kompatibilität (Drop-in-Ersatz, vorwärts wie rückwärts) und Inhaltsmigration sind zwei
unabhängige Fragen; MAJOR beantwortet die erste, `--no-migration`/ein Migrationsdokument die
zweite.
- Ein Grenzübertritt kann `kb/` völlig unangetastet lassen und trotzdem MAJOR sein - Katalog:
Update-Pfad, Release-Artefaktname, Paket-Import-Name, ein umbenanntes Kommando/Flag/Envvar,
die Shape einer maschinengelesenen Datei.
- Ein Breaking Change ist teuer (jede bestehende Instanz zahlt einmal, von Hand) und deshalb
genehmigungspflichtig: Bruch, Handarbeit je Instanz und Alternativen (Shim, aufschieben und
bündeln, aufspalten mit Deprecation-Fenster) vorlegen, dann Freigabe abwarten.
- Reine Prosa-Regeln drifted - deshalb wurde `--breaking` als Pflichtflag samt zweiter, von der
Migrationsprüfung unabhängiger `docs verify`-Prüfung eingeführt, nicht nur eine Textänderung.
## Aufgaben
Keine offenen Aufgaben aus dieser Sitzung - Issue #26 wurde in derselben Sitzung geschlossen,
mit Verweis auf Commit `31662dc` (`2.5.0`).
## Nicht übernommen
- Die vollständige Katalog-Tabelle und der `2.0.0`-Fallbeispiel-Text aus
`instructions/dev/version-parts.md` werden hier nicht wiederholt - die Datei ist die
autoritative Quelle (Instruktions-Layer, `manual`-artig durch die `instructions/dev/`-Grenze),
diese Source-Seite fasst nur zusammen, was zur Entscheidung führte.
- Der genaue Wortlaut der Tool-Fehlermeldungen (`version bump`-Refusals) steht im Transkript
selbst; hier nur die Regel dahinter.
## Verwandte Entities
- [[wikitool]]
- [[Chemenu]]
## Verwandte Concepts
- [[KB Stack Versioning]]
@@ -0,0 +1,67 @@
---
type: types/source.md
source_type: document
author: Torben
raw_files: [raw/documents/qmd - GitHub Repository.md]
source_url: https://github.com/tobi/qmd
source_language: en
date: 2026-09-02
tags: []
entities: [qmd]
concepts: []
summary: 'GitHub-API-Metadaten, package.json und README-Auszuege von tobi/qmd: TypeScript/Node/Bun statt Go oder Rust, BM25 (SQLite FTS5) plus Vektor-Suche (sqlite-vec) plus LLM-Reranking ueber node-llama-cpp.'
---
# Source: qmd - GitHub Repository
**Autor:** Torben
**Datum:** 2026-09-02
**Raw-Dateien:** raw/documents/qmd - GitHub Repository.md
**Typ:** Document
## Zusammenfassung
Direkter Abruf von `tobi/qmd` auf GitHub (Repo-Metadaten über die GitHub-API, `package.json`,
README-Auszüge), um eine bestehende, unsichere Behauptung auf der Entity-Seite [[qmd]] zu
prüfen. Die Seite trug seit ihrer Erstellung `Sprache: Nicht angegeben (wahrscheinlich Go oder
Rust)` bei `confidence: 0.85` - eine geratene Angabe mit einer Konfidenz, die für eine Vermutung
zu hoch war. Die GitHub-API meldet `"language": "TypeScript"`, mit einem byte-gewichteten
Sprach-Split von 1.597.168 B TypeScript gegenüber 200.880 B Python, 26.136 B Shell,
23.132 B JavaScript und weniger als 6 KB restlicher Sprachen - kein Go, kein Rust im Repository.
Das npm-Paket heißt `@tobilu/qmd`, installiert per `npm install -g` oder `bun install -g`,
ausgeführt per `npx`/`bunx` - keine Go-typische `go install`-Zeile, wie die alte Seite
behauptete. `package.json` listet `node-llama-cpp` (LLM-Inferenz), `sqlite-vec` (Vektorindex)
und `better-sqlite3` (SQLite-Bindung) als direkte Abhängigkeiten; das README bestätigt das
Zusammenspiel explizit: „QMD combines BM25 full-text search, vector semantic search, and LLM
re-ranking—all running locally via node-llama-cpp with GGUF models." Die Scoring-Tabelle des
Architektur-Abschnitts benennt zusätzlich `SQLite FTS5 BM25` als konkreten Volltext-Mechanismus,
nicht nur „BM25" allgemein.
## Kernaussagen
- Sprache: TypeScript (GitHub-API `language`-Feld, bestätigt durch die Byte-Gewichtung).
- Runtime: Node.js oder Bun, nicht Go/Rust - Installationsbefehle im README belegen das direkt.
- Such-Stack: SQLite FTS5 (BM25) + `sqlite-vec` (Vektor-Ähnlichkeit) + `node-llama-cpp`
(LLM-Reranking mit GGUF-Modellen) - deckungsgleich mit dem, was die alte Seite unter
„Such-Capabilities" bereits beschrieb, nur ohne konkrete Technologienamen.
- Enthält einen `@modelcontextprotocol/server`-Abhängigkeit, deckungsgleich mit der bereits auf
der Seite genannten MCP-Server-Funktion.
## Aufgaben
- [x] `kb/entities/tools/qmd.md`s Sprachangabe korrigiert und mit dieser Quelle belegt
## Nicht übernommen
- Der vollständige Funktionsumfang (Library-API, MCP-Server-Details, Tree-Sitter-basiertes
Code-Chunking, Konfigurationsdatei-Referenz, CLI-Unterbefehle) - nur das zur Korrektur der
Sprach-/Stack-Angabe Nötige wurde übernommen. Eine künftige Erweiterung der `qmd`-Seite um
diese Themen bräuchte einen neuen Abruf, nicht diese Quelle.
## Verwandte Entities
- [[qmd]]
## Verwandte Concepts
- None identified
+101
View File
@@ -0,0 +1,101 @@
# qmd - GitHub Repository (tobi/qmd)
> Source: https://github.com/tobi/qmd
> Retrieved: 2026-09-02, via `https://api.github.com/repos/tobi/qmd`,
> `https://api.github.com/repos/tobi/qmd/languages`, and the raw README/`package.json` at the
> `main` branch HEAD (`pushed_at: 2026-08-18T14:26:31Z` at retrieval time).
> Fidelity: the API JSON fields are quoted verbatim; the README/package.json excerpts below are
> quoted verbatim from the fetched files, trimmed to what backs the corrected claims on
> `kb/entities/tools/qmd.md` - not a full copy of either file.
## Repository metadata (GitHub API)
```json
{
"full_name": "tobi/qmd",
"description": "mini cli search engine for your docs, knowledge bases, meeting notes, whatever. Tracking current sota approaches while being all local",
"language": "TypeScript",
"html_url": "https://github.com/tobi/qmd",
"license": {"key": "mit", "name": "MIT License", "spdx_id": "MIT"}
}
```
Byte-weighted language breakdown (`/languages` endpoint):
```json
{
"TypeScript": 1597168,
"Python": 200880,
"Shell": 26136,
"JavaScript": 23132,
"Nix": 5984,
"Just": 1081,
"Dockerfile": 1074
}
```
## `package.json` dependencies
```json
{
"name": "@tobilu/qmd",
"dependencies": {
"@modelcontextprotocol/server": "2.0.0",
"better-sqlite3": "^13.0.3",
"fast-glob": "3.3.3",
"node-llama-cpp": "3.20.0",
"picomatch": "4.0.5",
"sqlite-vec": "0.1.9",
"tree-sitter-go": "0.25.0",
"tree-sitter-python": "0.25.0",
"tree-sitter-rust": "0.24.0",
"tree-sitter-typescript": "0.23.2",
"web-tree-sitter": "0.26.12",
"yaml": "2.9.0",
"zod": "4.2.1"
}
}
```
## README excerpts
Opening description (README.md, line 5):
> QMD combines BM25 full-text search, vector semantic search, and LLM re-ranking—all running
> locally via node-llama-cpp with GGUF models.
Install ("Quick Start" section):
```sh
# Install globally (Node or Bun)
npm install -g @tobilu/qmd
# or
bun install -g @tobilu/qmd
# Or run directly
npx @tobilu/qmd ...
bunx @tobilu/qmd ...
```
Scoring table (architecture section):
| Stage | Method | Formula | Range |
|---|---|---|---|
| FTS (BM25) | SQLite FTS5 BM25 | `Math.abs(score)` | 0 to ~25+ |
| Vector | Cosine distance | `1 / (1 + distance)` | 0.0 to 1.0 |
| Reranker | LLM 0-10 rating | `score / 10` | 0.0 to 1.0 |
Diagnostic command comment, confirming `sqlite-vec` is the vector index used at runtime, not
only a listed dependency:
```
# Diagnose the install (runtime, sqlite-vec, embedding fingerprints, GPU probe)
```
## Not extracted
The full README (1301 lines: library API, MCP server usage, tree-sitter code-chunking options,
config file reference, CLI subcommand reference) - only the excerpts backing the corrected
`qmd.md` claims (language, runtime, install method, search/rerank stack) were pulled. A future
qmd page expansion covering its library API or MCP server would need a fresh fetch, not a reuse
of this file's `## Not extracted` judgment.
@@ -0,0 +1,245 @@
# Conversation Transcript - MCP Read Server Implementation Session
> Source: Claude Code session (`claude-opus-5`), chemenu workspace
> Collected: 2026-09-02
> Participant: Torben
> Fidelity: **faithful summary transcript, not a verbatim log.** Torben's instructions are
> quoted verbatim; the assistant's design reasoning, code, and file contents are condensed to
> what a later reader needs to find the artifact - the artifacts themselves are the code, not
> this transcript. Command outputs shown (test counts, timings, the Mass-Update Gate's printed
> file list, gate refusal messages) are real, copied from the session. No credentials appeared.
> No subagents were used - all work in this transcript is first-hand.
> One of two transcripts cut from this session; the other covers fixing `kb/entities/tools/qmd.md`,
> a documentation defect noted but not fixed during this one.
Covers implementing the four-issue sequence held by #36 (the master issue for "path to the MCP
read server"): #34 (arm the Publish-Remote Gate), #33 (harden the read path), #31 (root
resolution + library boundary), #19 (the MCP server itself), plus the human-facing
`INSTALL-MCP.md` and filing #37 (the container-image follow-up) that #19's closure surfaced as
still missing. Four version bumps in sequence: 2.2.3, 2.2.4, 2.3.0, 2.4.0, then 2.4.1 for the
documentation follow-up. All five touched issues (#34, #33, #31, #19, #36) got closing comments;
#36 was left open pending #37.
---
## Turn 1 - `/stack-dev implementiere #36`
The `stack-dev` skill loaded, switching the session into tool-development rules (ordinary
software-engineering judgment on `tools/`, `types/`, `instructions/` - no source-binding, no
`raw/` citation needed for code).
Read issue #36 (the master issue) via `gitea-mcp`. It holds no implementation itself - a
sequence table (#34#33#31#19), a table of platform/auth/transport decisions already
made and closed to debate, a measured baseline (593 ms per `wikitool search`, broken down:
262 ms module import, 274 ms corpus parse, 7 ms ripgrep), ten verified findings backing the
design, and an explicit "not in scope" list (Go, deployment details, a write path, #15's
research capability). Read #34, #33, #31, #19 in full including comments to get the acceptance
criteria and the reasoning behind each.
Worked the sequence in order, starting a session-scoped `WIKITOOL_SESSION_ID`.
**#34 - Publish-Remote Gate.** `.wikitool-remotes.json` did not exist in this checkout; absence
means unrestricted per `git_publish.read_allowed_push_urls()`. Created it with `origin`'s exact
push URL as the sole allowed target. Counter-checked by adding a second git remote to a made-up
URL and running `publish --remote gatecheck`: refused with **exit 42**, working tree untouched,
before any fetch. Remote removed afterward.
`doctor`'s `check_publish_remotes()` reported only whether the *file* existed, not whether the
gate was actually armed. Rewrote its three outcome messages to lead with `Gate armed:` /
`Gate not armed:` explicitly - the single-remote-no-file case stays `OK` (nothing to protect,
and a FAIL would make the file mandatory through the back door) but now says outright that every
push target passes. The check had **no tests**; added three, one per state.
Bumped `--patch`**2.2.3**.
**#33 - Harden the read path.** Six findings, each addressed with a regression:
1. *ReDoS.* `search/ripgrep.py:_contains` passed `query.regex` to `re.search`. Deleted the
branch entirely rather than bounding it - `rg` already applies the pattern with a linear
engine before this function runs, so nothing is lost except the extra title/summary ranking
boost for a non-literal pattern (and summary/H1 lines are themselves counted by `rg`).
Regression test asserts both the timing (`< 0.5s` against `(\w+\s?)+$`) and the return value,
because a bound alone would pass with a merely-faster engine.
2. *No subprocess timeout.* Added `RIPGREP_TIMEOUT_SECONDS = 30.0`; `TimeoutExpired` translates
to the existing `RipgrepFailed` path.
3. *YAML aliases.* Frontmatter has no legitimate use for anchors/aliases, so they are refused
outright rather than budgeted, checked on the streaming *event* parse (`yaml.parse`) so the
check itself never triggers the expansion it is checking for - `*` is a necessary character
in any alias node, so its absence proves absence with zero parse cost. Constructed a
reproduction: 267 bytes of nested aliases compose into 672,603 nodes on traversal at
constant (0.2 ms) parse time, growth 9^n with nesting depth. Added a 64 KiB frontmatter size
limit and a `RecursionError` catch (PyYAML composes recursively; deep nesting is not a
`YAMLError`).
4. *`CSafeLoader` unused.* Switched to it with a `SafeLoader` fallback. Measured over this
corpus (176 pages, best of 5): **265 ms → 54 ms**.
5. *Silent frontmatter loss.* Decided: the read path now *reports* rather than swallows.
Unparseable YAML still degrades to `{}` for bulk operations, but the reason travels with it
(`Page.frontmatter_error`, set by a single parser shared between `read_page()` and
`frontmatter_error()` - previously two separate `safe_load` call sites that could describe
the same broken file differently). `search --json` now always carries an `unreadable` list
of `{path, reason}`; the table form writes the same lines to stderr. Added `read_page_strict()`
for the future ingest-quarantine path (#32), which must stop on unparseable input rather than
empty it.
6. *Corpus reparse per call.* New `chemenu/corpus_cache.py`: a `CorpusCache` object a caller
*holds* (not a module-global switch). The CLI holds none and is unaffected. Correctness
property, not just speed: **a dirty working tree is never cached** - if git cannot answer
whether the tree is clean, it is treated as dirty. `wikitool search` end-to-end (best of 5):
**593 ms → 347 ms**.
Bumped `--patch`**2.2.4**.
**#31 - Root resolution + library boundary.** `config.ROOT` and everything derived from it were
Python module constants, bound at import time from `Path(__file__).resolve().parents[2]` - so
`monkeypatch.setattr(config, "ROOT", tmp_path)` repointed `ROOT` but left `KB_DIR`/`RAW_DIR`
aimed at the real checkout, the exact failure class `raw_dir`'s own fixture docstring already
warned about one layer up.
Rewrote `config.py`: `resolve_root()` by precedence (explicit argument → `$CHEMENU_ROOT`
walk-up, the walk-up staying default so `tools/wikitool` is unaffected), and every derived path
resolved on *attribute access* via module `__getattr__` (PEP 562) rather than stored - so an
assignment onto `ROOT` (test or otherwise) is honored by every path under it, live. This
surfaced a subtler bug while fixing the first one: `monkeypatch`'s own undo mechanism reads the
old value (which resolves it) and writes it back as a real attribute on teardown, recreating the
stale binding the rewrite was meant to eliminate. Added `config.reset()`, called by the autouse
hermetic-environment fixture on both setup and teardown.
Split the CLI-coupled command modules into a pure core + thin adapter, three times over:
`search/service.py`, `lint_core.py`, `types_core.py` - none of them import `typer` or `rich`.
`commands/search.py`, `commands/lint.py`, `commands/types_cmd.py` became the terminal adapters,
re-exporting the same names so no existing import breaks.
New `chemenu/errors.py`: `ChemenuError``ValidationError` (also inherits `ValueError`, since
`PredicateError` already was one and callers catch it that way) / `BackendError`.
`PredicateError`, `FrontmatterError`, `UnknownBackend`, `RipgrepMissing`, `RipgrepFailed` moved
under this hierarchy.
New `chemenu/api.py`: `Corpus` class, the in-process entry point - takes a root, returns exactly
the CLI's `--json` shapes, raises instead of exiting, stamps every response with `commit`/
`as_of`. Wrote the acceptance test by monkeypatching `Path.read_text`/`Path.rglob` to fail on any
access under the real checkout root while a foreign root is set - this test could not have
passed before the lazy-resolution rewrite.
Two more accidental dependencies surfaced and were fixed: `TypeResolver.repo_root` was also
import-bound (fixed the same way, with `config.rooted()` as a process-wide context manager for
callers that reach `config` directly rather than taking a root argument - and `use_shipped_type_specs()`
added to the test fixtures that had been relying on it silently); and `search/registry.resolve()`
did not pass `kb_dir`/`root` through to the backend, so a caller pointing `run_search` at a
foreign corpus could still have `RipgrepBackend` read `config.KB_DIR` underneath it.
Bumped `--minor`**2.3.0** (new capability, backward-compatible).
**#19 - MCP read server.** New package `tools/chemenu/mcp/` (`server.py`, `__main__.py`).
Installed the `mcp` SDK (`mcp>=2.0`, discovered mid-session that this pulled in the v2 API -
`FastMCP` renamed to `MCPServer`, imported from `mcp.server.mcpserver`). Five tools over
`chemenu.api.Corpus`: `search`, `types`, `describe_type`, `lint`, `status` (`status` is
server-composed, not a wrapper - there is no `wikitool status` command to wrap). No write tool,
structurally: neither the server module nor `chemenu.api` imports anything under
`chemenu.commands`.
Both transports built and smoke-tested end-to-end against the real 176-page corpus: `stdio` via
a hand-written MCP client script, and `streamable-http` (host/port bound explicitly - the
default binds loopback, wrong for a container behind a proxy) via a subprocess + HTTP client
round-trip. `sse` deliberately not offered (superseded transport).
`ChemenuError` translated to the SDK's `ToolError` at the tool-call boundary (a deliberate
refusal whose message reaches the caller) rather than left to become an `UnexpectedToolError`
(a crash whose message stays server-side).
Found and fixed a stamping bug while writing the golden test: `_stamp()` was asking the cache for
the *current* revision after the load had already happened, so a caller with a perfectly clean
tree could see `"commit": null` if the cache's cached revision lagged. Fixed by threading the
revision the load actually returned through to the stamp.
Wrote `tools/chemenu/tests/test_mcp_server.py`: the golden test runs `wikitool ... --json` as a
subprocess against the same fixture tree (via `$CHEMENU_ROOT`) and asserts the server's
structured output matches field-for-field; a before/after test captures file size+contents,
`git rev-parse HEAD`, and `git status --porcelain` around all five tool calls to prove nothing
writes; a structural test imports the server module in a fresh interpreter and checks
`sys.modules` for absence of `chemenu.commands.*`.
Added `check_trace_destination()`: the server **refuses to start** if `WIKI_TRACE` is on and
would write into the served checkout's `reports/telemetry/` - the directory the sync's
`git reset --hard` is entitled to wipe. `WIKI_TRACE=0` or `WIKI_TRACE_DIR` outside the corpus
both satisfy it.
New `instructions/mcp-read-server.md` (agent-facing operating procedure), `tools/requirements-mcp.txt`
(the SDK's dependency, kept out of `requirements.txt` so a CLI-only instance need not install
pydantic/starlette/uvicorn/cryptography), a CI step to install it (so the golden test cannot
silently be skipped), and doc updates (`tools/CONTRACT.md`, `tools/README.md`, `README.md`).
Bumped `--minor`**2.4.0**.
Ran `docs verify`, `instructions verify`, and the full suite (**799 passed**) plus an
empty-machine run (`env -i ... pytest`) with the same result, before presenting the changeset.
`tools/wikitool publish` hit the **Mass-Update Gate**: 37 files, +3,037/-625 lines, exit 42.
Reproduced the full file-by-area breakdown verbatim in the reply (as the gate's message
requires - the output does not reach the user's screen on its own) and stopped without running
further commands, per the gate's instruction and AGENTS.md invariant 6.
## Turn 2 - "Freigegeben. Erstelle eine install-MCP.md für Menschen. Für traefik middleware verweise auf https://gitea.nehmer.net/torben/gitea-mcp-forward-auth ... Wir brauchen ein issue, der uns dicker Container für ein deployment erzeugt. Fasse dort schon mal zusammen, was du bisher weißt... Die ci scripte in https://gitea.nehmer.net/torben/gitea-mcp dürfte ein guter startpunkt sein."
Published with the token from the gate's own printed line
(`--confirm 46442f4419c1`) - commit `576df2c`.
Fetched both referenced repos via `gitea-mcp` (`get_repository_tree`, `get_file_contents`) to
ground the two follow-up documents in what actually exists rather than in assumption:
- `gitea-mcp-forward-auth`: a small Go Traefik ForwardAuth service - Bearer-token-vs-SHA-256-hash
checking, `AUTH_PROXY_TOKEN_HASHES_DIR`/`AUTH_PROXY_TOKEN_HASHES` config, `/healthz` always
open, demo token hashes checked into the repo, a registry-image test script with ephemeral
Docker/Podman login.
- `gitea-mcp`: its Dockerfile (multi-stage, `CGO_ENABLED=0`, distroless non-root, OCI labels) is
a usable pattern, but its release workflow pushes to **DockerHub** (`DOCKER_ORG: gitea`) - it
is a fork of the upstream project, so it is *not* a usable registry-push template for this
repo. Noted explicitly rather than silently copied.
Wrote `install-MCP.md` (later renamed, see Turn 3): six numbered steps (install dependency, run
stdio, wire a client, run streamable-http, put authentication in front, keep the corpus current
via polling), a runnable stdio verification script (executed for real before being written into
the doc, output: `Tools: [...]`, `Seiten: 176 | Commit: <sha>`), and a troubleshooting section
keyed to the server's actual error messages. Linked it from `INSTALL.md` and `README.md`, and
added `CHEMENU_ROOT`/`WIKI_TRACE`/`WIKI_TRACE_DIR` to `INSTALL.md`'s configuration table (both
had been in effect since earlier work but undocumented there). Added the file to `dist_cmd.py`'s
`ROOT_FILES` allowlist and verified with a real `dist export` that it ships.
Filed **issue #37** ("Container-Image für den MCP-Leseserver"): summarized what is already fixed
(startup command, required env vars, the ripgrep-in-the-image trap a naive `pip install`
Dockerfile would miss, where auth/rate-limiting belong) and the concrete templates found above,
then nine explicit open decisions (corpus baked into the image vs. mounted as a volume and
synced by a sidecar; who runs the sync; base image, since Go-style `distroless` does not carry a
Python interpreter or `rg`; how the version reaches the image; whether to build `arm64`;
**a healthcheck endpoint does not exist yet - the one item that implies new code in this repo**;
registry path; OCI labels; whether a smoke test against the built image is added). Labeled
`prio/2 size/M`.
Closed **#34, #33, #31, #19** with detailed comments each restating what was implemented against
the issue's own acceptance criteria, including the two accidental-dependency findings from #31
and the corrected alias-bomb numbers. Commented on **#36** with a summary table across all four
versions, noted its closing criterion (a consumer provably reaching the server through the
Traefik middleware) is not yet met - that needs #37's deployment - and proposed leaving #36 open
until then; also carried forward its two remaining loose ends (the wrong `qmd.md` language claim,
and the still-missing place for architecture decisions).
## Turn 3 - "Freigegeben. Nenne die Datei INSTALL-MCP.md all Caps sonst fahre mit dem Auftrag fort" (interrupting a `dist export` verification call)
Renamed `install-MCP.md``INSTALL-MCP.md`, fixed every reference (`INSTALL.md`, `README.md`,
`tools/chemenu/commands/dist_cmd.py`). Re-ran the full suite (**799 passed**) and `docs verify`,
bumped `--patch`**2.4.1**, wrote the changelog entry, published (commit `83018fc`). Saved a
feedback memory (`root-docs-are-all-caps.md`): root-level human docs in this repo are named in
ALL CAPS, and a new one has to be added to `dist_cmd.ROOT_FILES` or it silently does not ship.
## Outcome
- **Version:** 2.2.2 -> 2.2.3 -> 2.2.4 -> 2.3.0 -> 2.4.0 -> 2.4.1
- **Commits:** `576df2c` (2.4.0, the 37-file changeset cleared through the Mass-Update Gate),
`83018fc` (2.4.1, `INSTALL-MCP.md` and its wiring)
- **Tests:** 776 -> 786 -> 799 passed, green throughout, including on an `env -i` empty machine
- **Measured:** corpus parse 265ms -> 54ms; `wikitool search` end-to-end 593ms -> 347ms
- **Issues:** #34, #33, #31, #19 closed with detailed comments; #36 commented, left open pending
#37; #37 opened (`prio/2 size/M`)
- **CI:** `.gitea/workflows/ci.yml` updated to install `tools/requirements-mcp.txt`, otherwise
unchanged; not separately re-run in this session (publish triggers it)
- **Not done in this session, carried forward:** #37 itself (container image); #23 (env var
registration enforcement - `CHEMENU_ROOT` was added to `_WIKITOOL_ENV` by hand); the `qmd.md`
language-claim fix (separate transcript); an ADR-style home for architecture decisions
@@ -0,0 +1,128 @@
# Conversation Transcript - Private-Instance Merge Correction and Issue 30 Session
> Source: Claude Code session (`claude-opus-5`), chemenu workspace
> Collected: 2026-09-01
> Participant: Torben
> Fidelity: **faithful summary transcript, not a verbatim log.** Torben's questions and
> instructions are quoted verbatim; the assistant's reasoning is condensed. All shell output
> shown was produced by real, disposable test repositories built in a scratch directory during
> this session (not the chemenu repository itself) and is reproduced as run.
> No credentials appeared in the session.
> Third of three transcripts cut from this session; the first two cover the corpus/history work
> and the Publish-Remote Gate this one corrects a claim about.
Covers Torben asking what actually happens to the demo corpus in a private instance when the
upstream changes it - which exposed an unverified, incorrect claim the assistant had just
written into `instructions/private-instance.md` the same day. The claim was tested empirically,
found wrong, corrected in a released patch (2.2.1), and turned into a properly scoped, tested
merge procedure plus a new issue (#30) proposing the fix belongs in the tool itself, or that the
upstream should stop shipping content on the branch private instances pull machinery from.
---
## Turn 1 - "was passiert in der privaten instanz nach diesen schema wenn sich upstream der demo-korpus ändert?"
The assistant treated this as a request to verify, not answer from the existing text - the
previous transcript's `private-instance.md` had said a `git merge upstream/main` would see
*deleted-in-ours, unmodified-in-theirs* and resolve silently, which was written as an assumption
and never tested.
Built a throwaway upstream/private pair of git repos in the scratch directory to test it: private
instance deletes the demo corpus once, upstream then modifies one demo page, adds a new one, and
deletes another. Ran a plain `git merge upstream/main`. Result, measured rather than assumed:
- The upstream's **modified** page produced `CONFLICT (modify/delete)` and left the upstream's
version sitting in the working tree - resolving it the obvious way (`git add -A`) would commit
that version, bringing the deleted demo page back.
- The upstream's **new** page was staged silently. No conflict, no message.
- The upstream's **deleted** page (matching a page already deleted on the private side) produced
nothing - the one case that actually worked as claimed.
The middle case was the one the original claim missed entirely, and it is the dangerous one
because nothing announces it - a demo page the upstream adds simply arrives and starts showing
up in the private instance's own `lint`, `index`, `search` and `confidence decay`.
A second candidate fix was tested and also failed: a `.gitattributes` `merge=ours` driver for
`kb/**`/`raw/**`. Same three-scenario test showed the modify/delete conflict still occurred and
the new page still landed - the driver only resolves content conflicts on files both sides
touch, not modify/delete pairs or additions.
## Turn 2 - Fixing the instruction, in a released patch
`instructions/private-instance.md` was rewritten: the "Why a clone rather than a tarball"
section now states the measured three-case table instead of the wrong assumption. The update
procedure was rewritten to hold the merge open (`git merge --no-commit --no-ff`), force the
content stages (`kb/`, `raw/`) back to the pre-merge state while `HEAD` still points at the
commit before the merge, commit, and then run an explicit check
(`git diff --name-only $BEFORE HEAD -- kb raw` must be empty) - a check the reader cannot silently
skip past, unlike a claim they might trust. The bad advice in "Decision points" ("resolve a
conflict under kb/ as keep-deleted") was replaced, since that advice is exactly what leads
someone to `git add -A` the leaked content.
The corrected procedure was itself run against the same three-scenario test plus a fourth
(`raw/` alongside `kb/`), and against error paths (dirty working tree rejected, a second run
with nothing new to pull is a no-op) - all passing before publishing.
Version bumped `2.2.0 -> 2.2.1`. The changelog entry states plainly that the earlier text was
"nicht gemessen, sondern angenommen" (not measured, assumed) rather than framing it as a minor
wording fix. Published; tests green (752, unchanged in count - this was a documentation fix).
## Turn 3 - "stelle mal ein sauberes skripting hier exemplarisch dar. gibt es alternative setups? könnten wir die demo/test-seiten aus dem repo main irgendwie heraushalten? erstelle aus dem ganzen thema ein issue... aktualisiere/prüfe, ob #3 im gitea sauber ist. prüfe, ob wir sonst alle offenen enden verdrahtet haben."
Four asks in one message. Handled in order:
**A standalone script**, written and tested (not just described) against the same scenario:
rejects a dirty working tree, rejects a concurrent merge/rebase, holds the merge open the same
way the instruction's procedure now does, is a no-op on a second run with nothing new, and exits
non-zero with a rollback command if the post-merge diff check ever finds leaked content - the
check the manual procedure relies on a human to run, made unskippable.
**Alternative setups**, and the negative result from Turn 1 (`merge=ours` does not work) was kept
rather than omitted, since a rejected alternative is exactly the kind of finding this capture
procedure is meant to preserve.
**Whether the demo corpus can be kept out of `main` entirely** - assessed as the better fix in
principle: the private-instance script (and the instruction's manual procedure) treats a symptom
on every downstream instance, repeatedly; keeping content off the branch machinery is pulled from
would remove the need for either, by construction, for any new instance. Named as two variants
(a separate `demo` branch, or a separate demo repository) with the real cost stated rather than
glossed over - `main` becomes a content-free shell, which was the reason the corpus was kept in
the first place (as a walkable example), and `nightly.yml`'s corpus lint would need to move.
**Issue #30 opened**, carrying the measured table, the working script, both alternatives with
their tradeoffs, and the explicit note that it is not a duplicate of #28 but depends on it - #28
is why the corpus moves at all (demo and testbed sharing one `kb/`), #30 is what that movement
does to a downstream clone.
**Issue #3 checked against its own acceptance criteria** (all five, from the closed issue's own
text) rather than assumed closed-and-fine: a grep for the pre-rename name outside history/
changelog/transcripts, presence of a project entity page, the "Thoth" persona note in both
`SOUL.md` and its template, and no active technical identifier still using the old name. Found
clean - the handful of remaining mentions (`INSTALL.md`, two `kb/` pages) were all phrased as
history ("bis 2026-09-01", "damals"), which the criteria explicitly exempt.
**The wiring audit** turned up the two gaps described in the second transcript's closing turn
(`tools/CONTRACT.md` missing the new gate, `gates.md` not linking to `private-instance.md`) and
the stale project-page description - all three fixed in the same pass and published as 2.2.2,
verified with `docs verify`, `instructions verify`, `doctor` (16 checks OK), `lint` (clean but for
one pre-existing orphan page unrelated to this work), and the full test suite.
## Turn 4 - This capture
Torben asked for this session to be captured via `instructions/capture-session.md`, with an
explicit instruction to keep private and confidential material out - consistent with, not
separate from, everything the day's work had been about. Handled by generalizing every
infrastructure and personal-data reference that appeared during the day's audits rather than
reproducing the original identifiers, cutting the session into three topic transcripts (this
being the third), and filing this transcript's own existence as evidence rather than as new
open work - nothing in this turn required a new issue.
## Outcome
- **Version:** 2.2.0 -> 2.2.1 -> 2.2.2
- **Commits:** `private-instance.md` correction (2.2.1), documentation wiring + project-page
rewrite (2.2.2, shared with the second transcript's closing turn)
- **Tests:** 752, green throughout
- **Issues:** #30 opened; #3 verified against its own acceptance criteria and confirmed clean,
no action needed
- **CI:** green after each publish in this transcript's scope
@@ -0,0 +1,208 @@
# Conversation Transcript - Public Release, Corpus Purge and History Squash Session
> Source: Claude Code session (`claude-opus-5`), chemenu workspace
> Collected: 2026-09-01
> Participant: Torben
> Fidelity: **faithful summary transcript, not a verbatim log.** Torben's instructions and
> decisions are quoted verbatim; the assistant's reasoning and the exploration agents' findings
> are condensed. Command outputs shown are real, but every private hostname, IP address and
> personal detail that appeared during the audit has been **generalized rather than repeated** -
> reproducing them here would undo the point of the session. No secret values (tokens, keys)
> ever appeared.
> **Several passages are second-hand.** Three `Explore` subagents (private-data inventory,
> distribution mechanism, git-history audit) and three forked debate agents (hard-separation,
> shared-upstream, devil's-advocate positions) ran in the background; their findings are
> summarized here, not reproduced verbatim, and are named as such at each point they inform a
> decision.
> One of three transcripts cut from the same session; the other two cover building the
> Publish-Remote Gate and the private-instance procedure, and correcting a bug found in that
> procedure afterward (issue #30).
Covers turning the chemenu wiki stack from a private testbed into a publishable repository:
deciding what to delete versus anonymize, squashing the git history, choosing a license, and
verifying the result anonymously end to end. Resulted in the corpus purge (108 pages, ~90 raw
files removed), a single-commit public history, `LICENSE`/`LICENSE-CONTENT`/`NOTICE`, the
`dist export` leak canary (2.1.0), a CI fixture fix (2.1.1), and the repository going public on
Gitea.
---
## Turn 1 - "ich möchte die kb hier bereinigen, es soll nur noch test content sein, ich möchte das repo veröffentlichen..."
Torben's instruction, in substance: clean the KB down to test content, publish the repository,
falsify or remove IP addresses, keep the repo usable as a demo and a testbed going forward; he
would rebuild his real private instance separately. He asked for tips on three specific
questions - how a private fork could pull upstream fixes back, how to set up that private copy
and where to develop the stack, and whether a `kb/manual/` should ship with the distribution so
it documents itself - explicitly requested "mindestens ... teufels anwalt modus" and a small
multi-agent debate, and flagged himself as new to packaging/distributing an LLM application.
The session started in plan mode. Three `Explore` subagents ran in parallel:
- **Private-data inventory** - catalogued what in `raw/` and `kb/` was hard-private (a real
home-lab cluster's bootstrap manual, several personal notes, a 2.46 MB conversation transcript
full of infrastructure references), what was anonymizable (internal hostnames, private IP
ranges), and what was legitimately generic (the stack's own concepts, technologies, tools).
- **Distribution mechanism** - read `dist_cmd.py`, `INSTALL.md` and the version-check machinery;
confirmed `dist export` already ships no `kb/`/`raw/` content and that the update feed's
default URL pointed at the (then-private) origin repo.
- **Git-history audit** - measured `.git` size, largest blobs, deleted-file history, and
concluded the tree was "safe to publish" - a conclusion the session later disproved, see Turn 4.
## Turn 2 - The debate
Three forked debaters argued from the same context:
- **A (hard separation):** delete and rebuild the corpus rather than anonymize, because a page
title is the wiki's only identifier and lives in wikilinks/citations/frontmatter across the
corpus - renaming is the full page-lifecycle procedure, deleting is a supported command.
Squash history rather than selective `filter-repo`, because the history audit itself had just
been shown to under-report risk. No shared git ancestry between the public repo and the
private instance, ever.
- **B (shared upstream):** clone-with-`upstream`-remote instead, because the tarball update path
in `INSTALL.md` is `cp -r` with no three-way merge; argued the leak risk from shared history is
a mechanism problem (a remote allowlist), not a discipline problem, and that most of the
corpus (the ~190 generic pages) is worth keeping as real test data rather than deleting it all.
- **C (devil's advocate):** attacked both positions' shared premises - "just a testbed" was
false (`USER.md` is tracked and carries real personal data), "publish the repo" was
underspecified (which host, and what breaks), there was no `LICENSE` at all, a shipped manual
would decay under `confidence_decay.py` and mislead a new instance, and demo/testbed are
conflicting goals for one `kb/`.
The assistant's synthesis kept A's "delete, don't anonymize" and "squash the history" (C's
finding that the audit under-reported risk made this decisive), adopted B's clone-with-`upstream`
model but **conditioned it on a remote-allowlist gate existing first** (addressing C's leak
concern with code, not discipline), rejected the shipped manual per C's decay argument, and
decided the surviving corpus should lean toward the stack's own self-documentation rather than
Torben's real infrastructure - which the actual corpus math later validated (Turn 5).
## Turn 3 - Clarifying questions and plan approval
The assistant asked Torben to choose among concrete options (`AskUserQuestion`) rather than
picking on his behalf:
- **Publish target:** "Gitea öffentlich schalten" (chosen) over mirroring to GitHub or keeping
Gitea private with a GitHub mirror - smallest change, CI/release feed stay correct as-is.
- **Corpus:** "Chirurgisch löschen (Empfehlung)" (chosen) over full synthetic rebuild or
anonymization.
- **Private instance:** "Clone mit upstream, Gate zuerst (Empfehlung)" (chosen) over hard
separation or a later migration.
- **License:** first pass "Was wäre CC-BY-4 für content und eine GPL oder LGPL fürs Tooling?
Berate mich hier noch mal" - the assistant advised against LGPL (no library-linking use case
here to justify the fetch-only variant), and named the real choice as GPL vs. **AGPL**, because
issue #19 (an MCP frontend for `wikitool`) points toward running the stack as a network
service, which is exactly the gap GPL leaves and AGPL closes. Torben chose **AGPL-3.0 +
CC-BY-4.0**.
The written plan covered: publish blockers (license, `dist export` allowlist, `INSTALL.md`
token wording), the corpus purge, the history squash, the private-instance model with the gate
as a precondition, why the manual idea was declined, and a leak-canary for `dist export`. Torben
approved it via `ExitPlanMode` without further changes.
## Turn 4 - Executing the license and leak-canary work (2.1.0)
Before touching content: a full repo bundle backup outside the tree, verified by `git bundle
verify` and a commit-count comparison (150 commits, all 15 tags present).
- Fetched the AGPL-3.0 text from `gnu.org` (not reconstructed from memory - a license must be
the authoritative text) and copied the CC-BY-4.0 text from the vendored `commonplace`
submodule, per the license each already carries.
- Added `LICENSE` (AGPL - deliberately the AGPL rather than a separate `LICENSE-CODE`, because
that is the file a forge reports for the repository, and under-noticing a copyleft obligation
harms a reader in a way over-noticing does not), `LICENSE-CONTENT` (CC-BY), `NOTICE` (the
license boundary and the `commonplace` attribution CC-BY requires).
- `dist_cmd.py`: added the three license files to `ROOT_FILES`, and a separate
`REQUIRED_ROOT_FILES` check that **fails the export** if they are missing - every other
`ROOT_FILES` entry is copied `if source.is_file()` and silently skipped otherwise, which is
wrong for a license (a distribution shipping AGPL code with no license text is a violation the
moment it is published).
- Added `find_leaks()`: a structural (not text-pattern) check of the finished export plan against
personalization files, `instructions/dev/`, `kb/` pages and `raw/` sources - rejected a
hostname/IP text-scan approach, because the project's own host legitimately appears in
`INSTALL.md` and `version.py`, so such a scan would either whitelist the string it's looking
for or false-positive on every export.
- Six new tests in `test_dist_cmd.py`; full suite green; version bumped `2.0.0 -> 2.1.0`.
## Turn 5 - The corpus purge
A workshop (`work/publish-cleanup`) was opened per `work/CONTRACT.md`, because ~103 planned
`wikitool rm` calls exceed a single iteration-budget unit.
Analysis found the plan's estimate wrong in a way worth recording: not ~35 pages would become
sourceless after deleting the obviously-private ones, but **75**. Splitting them: 28 were the
user's own infrastructure (unambiguous), and **40 carried generic technology titles but were
written entirely around the user's own cluster** (a storage page explained itself via one
specific storage class name, a Kubernetes page via the user's own network ranges) - not
reusable pages with a private example, but private documentation with a generic heading. The
remaining 7 were genuinely clean in body text but sourced from the user's personal
document-processing pipeline.
Presented to Torben as a three-way choice; he chose **"Alle 75 löschen"** over keeping the
clean 7 (via provenance reclassification) or rewriting the 40 as vendor-neutral pages. The
result: 152 pages plus 25 sources remained by design - the stack's own self-documentation
(gates, lint, versioning, search, the wiki pattern itself), which the earlier debate had argued
for without knowing this would be the actual outcome.
Execution: `wikitool rm --page <title> --yes` per page across three units (27 sources, 28
infra, 48 entangled), 90 raw files removed including the full cluster bootstrap tree and the
2.46 MB transcript. `wikitool rm` was found to de-link only mechanically (frontmatter refs and
whole-line link bullets) and to deliberately leave inline prose wikilinks and plain-text
mentions standing - roughly 20 pages needed manual follow-up to actually remove the remaining
references, which `lint` and a targeted `rg` scan surfaced.
Also reset: `kb/log.md` (116 private references, no regenerator - reset to the
`dist_templates/log.md` starting state, a deliberate one-time stack-dev operation, not a
hand-edit of a generated file), `USER.md`/`SOUL.md` rewritten as an explicit demo-operator
persona rather than Torben's real profile.
## Turn 6 - Squashing the history
Backup re-verified before the destructive step. An orphan root commit was built from the purged
working tree, `main` reset onto it, all 15 tags deleted (locally, then individually on the
remote - a batch refspec push failed silently), `git reflog expire --expire=now --all` and
`git gc --prune=now --aggressive`, then a `--force` push.
**Verified rather than trusted:** a fresh clone attempted `git fetch --depth=1 origin
<old-head-sha>` immediately afterward and succeeded - the objects were still reachable through
Gitea's own reflog on the bare repository, contradicting the earlier belief that force-pushing
was sufficient. The rejected shortcut here was accepting Gitea's own scheduled cleanup jobs as
proof; Torben ran them once, and the same fetch test still succeeded afterward. Only a direct
`reflog expire` + `gc --prune=now` run on the bare repository itself (which Torben ran, having
shell access the assistant did not) closed it - reverified by the same fetch test returning
`not our ref`.
A second leak was found by scanning **hidden directories** (`.gitea/`, `.github/`, `.vibe/`),
which the earlier `rg` sweeps had silently skipped without `--hidden` - two references to an
internal CI branch name in `.gitea/workflows/ci.yml`, fixed and amended into the squashed
commit before the first push.
## Turn 7 - Public switch, CI fixture bug, and verification
Torben switched the Gitea repository to public. Verified anonymously (no token, no SSH key):
repository API reports `private: false`, the release feed serves the latest tag, and a clean
`git clone` over HTTPS from an empty directory succeeds. A further leak scan of that anonymous
clone found one remaining hit: the 2.1.1 changelog entry (see below) had *listed* the private
fixture names it replaced, which put them back into the public history it was announcing the
removal from. Corrected in the same commit that introduced it.
Separately, the first CI run on the squashed history failed on a test unrelated to the corpus
purge in content but caused by it: `test_legacy_source_pages_flags_url_and_directory` checks
`(config.ROOT / legacy).is_dir()` against the **real** repository root rather than the test
fixture's own tree, and had only ever passed because this checkout happened to have a
`raw/documents/` directory - which the corpus purge had just emptied. Git does not track empty
directories, so the directory vanished from CI's checkout and stayed in the local one: green
here, red there. Reproduced locally by removing the directory and rerunning; fixed in the
`raw_dir` fixture (`conftest.py`) rather than the one test, matching the reasoning already
recorded for a prior, similar case (Gitea #8). Several other test fixtures still used the
user's real system names and were renamed to generic, unrelated placeholders
alongside. Published as 2.1.1 after Mass-Update Gate clearance from Torben.
## Outcome
- **Version:** 2.0.0 -> 2.1.1
- **Commits:** license/leak-canary work, corpus purge (multiple), history squashed to one root
commit, CI fixture fix
- **Tests:** 630 -> 752, all green
- **Issues:** none opened in this transcript's scope (see the other two transcripts for #27-#30)
- **CI:** green on the squashed history after the fixture fix; release `v2.1.1` built and
anonymously downloadable
@@ -0,0 +1,154 @@
# Conversation Transcript - Publish-Remote Gate and Issue Triage Session
> Source: Claude Code session (`claude-opus-5`), chemenu workspace
> Collected: 2026-09-01
> Participant: Torben
> Fidelity: **faithful summary transcript, not a verbatim log.** Torben's instructions are
> quoted verbatim; the assistant's design reasoning is condensed. Command outputs and test
> results shown are real. No credentials appeared in the session; where a private hostname
> could have been used as a code example, a generic placeholder is used instead, matching what
> was actually committed.
> Continues directly from "Public Release, Corpus Purge and History Squash Session" (same day).
> Two of three transcripts cut from this session; the third covers a bug found afterward in the
> procedure this one builds (issue #30).
Covers building the Publish-Remote Gate (a third code-enforced gate, alongside the Mass-Update
and rebase-review gates), writing `instructions/private-instance.md`, updating six open Gitea
issues for the rename and the new architecture, opening three new issues (#27, #28, #29), and
deleting the Gitea Actions run history at Torben's request. Resulted in version 2.2.0, then a
documentation-only 2.2.2.
---
## Turn 1 - "1. remote allowlist. 2. unterbreche und warte auf mich, wenn ich gitea scharf schalten soll... 3. setup a) meiner lokalen dev-umgebung"
Torben asked for three things in one message, plus: "parallel issuen anlgen und aktualisieren
nach eigenem ermessen, prüfe auch vorhandene, evtl. müssen wir hier auch noch tätig werden?"
**The gate.** Designed to pin the **resolved push URL**, not the remote name - a name-based
allowlist would pass a `publish` whose `origin` had been repointed, which is the exact failure
it exists to catch. `git remote get-url --push <remote>` is read at check time so a `pushurl`
override is respected. The file (`.wikitool-remotes.json`) is gitignored and per-checkout, for
the same reason `ENVIRONMENT.md` is: two clones push to two different places, and a committed
copy would tell a private clone the public upstream is a legitimate target for its own content.
Absence means unrestricted, matching the pattern of the other optional per-checkout files;
`doctor` reports the state and WARNs only when a checkout has more than one remote and no
allowlist.
**Deliberately no `--confirm` token**, unlike the other two gates. Their question ("is this
change right?") is answerable per changeset; this gate's question ("does this content belong in
that repository?") is a standing property of the checkout, so the only way past it is the user
editing the file themselves - an agent editing it to clear a refusal would be opening a gate on
its own initiative, which the repository's own rules forbid.
Implementation: `config.PUBLISH_REMOTES_FILENAME`, `read_allowed_push_urls()` /
`push_url_for()` / `publish_remote_refusal()` in `git_publish.py`, checked before the reconcile
step in `publish_command` (before any network contact, so a refused publish never even fetches
from the wrong place), `doctor.check_publish_remotes()`, twelve new tests covering the URL-vs-name
distinction, `pushurl` precedence, a broken/missing/empty allowlist, and that no flag exists to
bypass it. `instructions/gates.md` and `AGENTS.md` updated to describe a third gate. Version
bumped `2.1.1 -> 2.2.0`; the Mass-Update Gate itself fired at 10 files and Torben cleared it with
the printed token.
**`instructions/private-instance.md`** (new): the procedure for cloning with the public repo as
`upstream`, arming the gate *before* the first `publish` (not after - the assistant stressed this
ordering explicitly, since a gate added later leaves the earlier window open), taking the write
credential away from the private clone as a second, independent control, and where stack
development happens ("in the public repo, not here" - not a preference but a structural fact,
since `instructions/dev/` does not survive `dist export`). **This file's description of what a
`git merge upstream/main` actually does to the content stages was wrong as first written** - see
the third transcript for the correction.
**The Gitea admin cheat-sheet** (delivered as a reply, not committed): what to check before
flipping the repo public (`DISABLE_REGISTRATION`, `REQUIRE_SIGNIN_VIEW`, rate limits, the
Actions runner's network exposure), and the follow-up steps (`INSTALL.md`, anonymous release
check, a clone test). Also flagged that the open issues would go public with the repo, which is
what motivated the issue-triage pass below.
## Turn 2 - Issue triage
Read the labels (`prio/1..3`, `size/XS..L`) and the open issues. Several pre-dated the
`llm-wiki-test1` -> `chemenu` rename (issue #3) and still named the old package path
(`tools/wiki_tools/...`) or the old repository name in code examples:
- **#6** title corrected in place (`wiki_tools` -> `chemenu` path).
- **#7** (`dist upgrade`) commented: the "origin repo is private" fallback it describes no
longer applies once the repo is public, and its urgency for *this* instance specifically
dropped, because the private instance now takes updates via `git merge upstream/main` (real
three-way merge) rather than the tarball-copy path the issue was written against - it remains
the right design for any instance without shared git history.
- **#10** (coverage reporting) commented: step 1 is done (1.8.1 shipped `pytest-cov` without a
failure threshold), the test count referenced is stale (630 -> 752, and the gap is itself
evidence for the issue's own argument - the raw_dir fixture bug from the first transcript), and
one code example named a private CI branch and needs neutralizing before being read publicly.
- **#4, #5** flagged as needing the same path correction but not rewritten in this pass.
**Three new issues opened**, each traced to a decision made and then set aside earlier in the
day rather than invented fresh:
- **#27** - the shipped-manual idea the debate rejected (first transcript, Turn 2) needs a
decay/lint exemption for distributed pages before it becomes buildable at all; names the exact
blocker (`confidence_decay.py` has no exemption path) and the two things still undecided (how
a shipped page is recognized, what exactly is exempted).
- **#28** - the demo-vs-testbed tension the debate argued through (assistant recommended
testbed, C objected) is not resolved, only decided for now; proposes a curated fixture corpus
as the eventual answer and names why (`kb_dir`/`raw_dir` fixtures already cover the small case,
this is about the dense, interconnected one).
- **#29** - catalogs which open issues still carry pre-rename paths and a private hostname
(#10's CI example), separating "pointer, fix it" references from "citation of what was true
then, date it" references rather than blanket-editing everything.
## Turn 3 - Public verification and the actions-history request
After Torben confirmed the repository was public, verification ran anonymously: repo API
(`private: false`), release feed (latest tag served without auth), a clean HTTPS clone from an
empty directory (three commits, two tags, all three license files present, 170 pages, no leak
hits including in hidden files), and the full `INSTALL.md` release-tarball path end to end -
checksum verified, license files present in the unpacked tree, `kb/`/`raw/` empty as designed,
`instructions/dev/` absent.
Torben: "an der Stelle: wenn du per mcp die action runner historie löschen kannst, leere sie
bitte komplett, da finde ich keinen knopf für und dort dürften noch alte tarballs herumgeistern."
The assistant checked: the Gitea MCP server exposes `dispatch`/`cancel`/`rerun` for Actions runs
but no delete, and the delete route Gitea's own API defines (`DELETE
/api/v1/repos/{owner}/{repo}/actions/runs/{run}`) requires a token the assistant did not hold.
Findings reported instead of silently working around the missing capability: **zero artifacts**
existed already (confirmed via the anonymous artifacts API), but an anonymous SPA data endpoint
the run pages call was readable and returned old commit titles from the squashed-away history -
no hostnames or personal data among the six reachable runs, but development history the squash
was meant to remove. A `curl` loop using a write-scoped token was handed to Torben to run
himself, rather than asking for a token in the session.
Torben ran it and reported back ("die action-historie habe ich jetzt per api gelöscht") in the
same message as "korrigiere die install.md wie vorgeschlagen." Both handled: the anonymous SPA
endpoint was re-tested across all previously-reachable run numbers and returned nothing for any
of them, closing the loop; `INSTALL.md`'s "repo is currently private, you need a token" language
was removed from three places (the download command, the configuration table, a dedicated
paragraph - the paragraph rewritten to explain the token is still needed **for a private fork**,
not for this repository), "Weg C" was reworded from "Torben's personal wiki" to a description of
the testbed/demo nature of the corpus, and a new "Weg D" documented the private-instance path
with a pointer to `private-instance.md`. No version bump (prose only).
## Turn 4 - Wiring audit (from the third transcript's closing turn, included here for the gate/issue thread)
A later pass (opened by a different question, documented fully in the third transcript) checked
whether the gate work here was actually cross-referenced: `tools/CONTRACT.md` did not mention
the Publish-Remote Gate at all despite it being `publish`'s third exit-42 path - fixed in both
the command-table entry and the error-contract entry, since a tool's error contract is exactly
where a caller learns what a given exit code means and whether retrying is safe. `gates.md` did
not link to `private-instance.md`, the procedure it exists for - fixed. The project's own KB
page (`kb/entities/projects/Chemenu.md`) still described the instance as a personal wiki with no
mention of the license or its public, testbed-and-demo status - rewritten, keeping the
historical note about the pre-restructuring `AGENTS.md` but dating it explicitly rather than
stating it as a current fact. Published as 2.2.2.
## Outcome
- **Version:** 2.1.1 -> 2.2.0 -> 2.2.2
- **Commits:** Publish-Remote Gate + `private-instance.md` (2.2.0, Mass-Update Gate cleared),
`INSTALL.md` correction (unversioned prose), documentation wiring + project-page rewrite
(2.2.2)
- **Tests:** 752, all green throughout (12 new for the gate)
- **Issues:** #6 title fixed; #7, #10 commented; #27, #28, #29 opened
- **CI:** green on every push in this transcript's scope; Gitea Actions run history removed by
Torben via the API, verified anonymously empty afterward
@@ -0,0 +1,218 @@
# Conversation Transcript - Version Part Nomenclature and Breaking Change Gate Session
> Source: Claude Code session (`claude-sonnet-5`), chemenu workspace
> Collected: 2026-09-02
> Participant: Torben Nehmer
> Fidelity: **faithful summary transcript, not a verbatim log.** The user's instructions and
> clarifications are quoted verbatim; the agent's own reasoning and file-reading steps are
> paraphrased; tool output blocks (`docs verify`, `pytest`, `version bump --dry-run`, the
> Mass-Update Gate refusal) are real, copied from the actual run, not reconstructed.
> No second-hand material - no subagent was used.
> No credentials appeared.
> Single topic, not cut.
Closes Gitea issue #26 (already closed in-session) with `2.5.0`. Commit `31662dc` on `main`.
Covers picking the version part for a stack change, and adding a user-approval gate for
breaking changes to `stack-dev`.
## Turn 1 - `/stack-dev kümmere dich um #26`
The user invoked the `stack-dev` skill with the argument `kümmere dich um #26`. The agent read
issue #26 via `gitea-mcp` (`torben/chemenu#26`, no comments yet).
**The issue's own account** (quoted from its body, since it is the source of the whole
session): the stack's documentation conflated two questions when choosing a version bump part -
"Muss der Korpus migriert werden?" and "Ist der Wechsel rückwärtskompatibel?" - and every place
an agent would consult before a bump stated only the first:
| Ort | Wortlaut |
|---|---|
| `instructions/dev/stack-dev/SKILL.md`, Schritt 3 | `--major` ⇔ "Existing content must be migrated" |
| `tools/chemenu/version.py`, module docstring | "from `1.0.0` on the same rule reads as the familiar 'MAJOR means migration'" |
| `INSTALL.md` § Version und Updates | "ab `1.0.0` liest sich dieselbe Regel als das gewohnte 'MAJOR heißt Migration'" |
Only `version bump --help` had it right ("A bump that crosses the compatibility boundary
additionally requires a migration document ... or `--no-migration`") - the one place read only
after the decision is already made.
The issue's worked example was the `2.0.0` rebranding (`torben/llm-wiki-test1`
`torben/chemenu`): no page in `kb/` changed, so the first attempt at that bump was `1.9.0`. Three
things broke anyway - the release-feed URL baked into every existing instance's
`.wikitool-release.json` (unrepairable by hand, per invariant 1), the release artefact's
filename, and the Python package's import name. The correct bump was `--major --no-migration`.
The issue's "Was 'fertig' heißt" checklist asked for: rewording the three prose locations to
lead with compatibility rather than migration, a decision aid enumerating which kinds of break
force MAJOR even with `kb/` untouched, and a check that the three locations do not end up
contradicting each other (invariant 8). It explicitly scoped out teaching `docs verify` to judge
*whether* a bump chose the right part - "das ist eine Urteilsfrage und kein hartes Orakel."
The agent read the three prose locations plus `tools/chemenu/commands/version_cmd.py` (the
existing `--no-migration` enforcement) and `docs_verify.py` (`check_migration_for_boundary`), to
scope the fix before writing anything.
## Turn 2 - scope questions, then the user's clarification
Before writing, the agent asked two `AskUserQuestion` questions:
1. Whether the "Breaking Change" marker and approval requirement should be prose-only or also
enforced in the tool. Options offered: "Prosa + Marker im Code (Empfehlung)" vs. "Nur Prosa".
**User picked the enforced option** ("Prosa + Marker im Code (Empfehlung)") - a
`--breaking` flag on `version bump`, checked by `docs verify`, on the reasoning that a
pure-prose rule is exactly what drifted in the first place.
2. Whether the pre-bump approval step should read like one of the codebase's actual gates
(`instructions/gates.md` language: "anhalten ... auf ausdrückliche Freigabe warten") or as a
softer "Decision points" entry. **User picked the softer framing** ("Als Entscheidungspunkt")
- explicitly *not* gate language, to avoid diluting the weight of the three real,
code-enforced gates (Mass-Update, Publish-Remote, Iteration Budget) by association.
The user then added the clarification that drove the rest of the session, quoted in full:
> "genaue nomenklatur für major version change: die neue version ist kein drop-in replacement.
> Sobald irgendwie Hand angelegt werden muss, sei es durch den user oder durch ein
> Migrationsscript, ist es ein major version change. selbiges gilt, wenn ein update nicht
> rückgängig gemacht werden kann (Szenario: User aktualisiert den Stack von vA nach vB, vB hat
> beim User ein Problem, kann jetzt vA nicht mehr einfach so als Downgrade installiert werden,
> ist es ein breaking change)
> in allen Fällen muss bei einem Major version change ein "Breaking Change" vermerkt werden.
> breaking changes sind damit teuer. passe stack-dev so an, dass in diesen Fällen zwingend der
> user informiert, Alternativen aufgezeigt und eine freigabe eingeholt wird."
This sharpened the issue's "compatibility" question into a concrete two-sided test - **forward**
(can the new version be copied over with zero hand-work, by user or script) and **backward** (can
the old version still be reinstalled afterwards) - and made explicit that either half failing,
independent of `kb/`, is what MAJOR actually means.
## Turn 3 - what was written
**`instructions/dev/version-parts.md` (new).** The single source for the rule. Structure:
the two-sided drop-in test (step 1); a catalogue of breaks that cross the boundary with `kb/`
untouched - update path, release-artefact name, package import name, a renamed
command/flag/envvar, the shape of a machine-read file like `.wikitool-kb.json` (step 2); the
plain compatible-part table for patch/minor (step 3); the mandatory stop-and-inform-the-user
step before any boundary-crossing bump, with three named alternatives (avoid via a
compatibility shim, defer and batch with the next crossing, split into a compatible half now
and a deprecated-then-removed half later) plus a recommendation (step 4); `--breaking` usage
(step 5); the pre-existing migration-document-or-`--no-migration` step, now explicitly
independent of compatibility (step 6); writing the entry body (step 7). Closes with the `2.0.0`
case study as the worked example, matching the issue's own account.
Placed under `instructions/dev/` (not linked from any distributed artifact) because
`tools/wikitool dist export` prunes that directory wholesale - the agent's first draft linked to
it from `tools/CONTRACT.md` and the `version.py` docstring, which `tools/wikitool instructions
verify` correctly rejected:
```
ERROR Instruction layer issues:
- version-parts.md: lives under instructions/dev/ but is referenced from
outside it and outside a dist:strip block - `dist export` removes
instructions/dev/ wholesale, so that reference would dangle in a distributed
instance. Remove the reference, or wrap it in a <!-- dist:strip-start/end -->
block if it belongs only to this dev instance.
```
**Rejected approach:** wrapping the reference in `<!-- dist:strip-start/end -->` markers so it
would still resolve in this repo. Not used - the agent instead rewrote the three shipped
locations (`tools/CONTRACT.md`, `version.py` docstring, `version_cmd.py` docstring) to state the
short form of the rule standalone, with no pointer to the dev-only file, since a shipped
instance never has it to point to.
**`instructions/dev/stack-dev/SKILL.md`.** Step 3's table changed from "Existing content must
be migrated → `--major`" to "Not a drop-in replacement ... → `--major`", with a pointer to
`version-parts.md` for the full test and catalogue. A new "Decision points" entry: if a change
turns out not to be a drop-in replacement, stop - do not bump across the boundary on the
session's own initiative; show the user the concrete break, what each instance must do, and the
three alternatives from `version-parts.md` step 4, then wait for a go-ahead. Written in the
softer "Decision points" register per the user's second answer above, not gate language.
**`tools/chemenu/version.py`.** Module docstring reworded: "MAJOR means migration" → "MAJOR
breaks", with a new paragraph stating the two questions are independent and naming both markers.
New constant `BREAKING_CHANGE_MARKER = "**Breaking Change:**"`, alongside the existing
`MIGRATION_NONE_MARKER`. `insert_changes_entry()` gained a `breaking_reason` parameter, writing
the `**Breaking Change:**` line *before* the migration line - the break is what an operator acts
on first.
**`tools/chemenu/commands/version_cmd.py`.** `bump` gained `--breaking TEXT`. Refused on a
non-boundary-crossing bump; required on a boundary-crossing one, with a refusal message that
also second-guesses the part choice itself: *"If that sentence is hard to write because nothing
actually breaks - no hand-work on update, and the old version can still be put back - then the
bump is probably not --major."* Verified live:
```
$ tools/wikitool version bump --major --title "smoke" --dry-run
ERROR 2.4.1 -> 3.0.0 crosses the compatibility boundary, so it is not a drop-in
replacement - re-run with --breaking "<what stops working, and what an instance
must do about it>".
If that sentence is hard to write because nothing actually breaks - no
hand-work on update, and the old version can still be put back - then the
bump is probably not --major.
```
```
$ tools/wikitool version bump --minor --title "smoke" --breaking "x" --dry-run
ERROR --breaking only applies to a bump that crosses the compatibility boundary;
2.4.1 -> 2.5.0 does not.
```
**`tools/chemenu/commands/docs_verify.py`.** New `check_breaking_change_for_boundary()`,
independent of the existing `check_migration_for_boundary()` on purpose - the two check
different things (did the corpus move vs. was the operator told the swap isn't drop-in), and a
crossing with `kb/` untouched can satisfy the first while failing the second. Wired into
`verify()` alongside the existing check.
**Tests.** `test_version_cmd.py`: nine existing calls updated for the new `breaking=` parameter,
plus three new tests (`test_a_boundary_crossing_bump_without_breaking_is_refused`,
`test_breaking_records_what_stops_working_in_the_changelog` - including the ordering assertion
that `BREAKING_CHANGE_MARKER` precedes `MIGRATION_NONE_MARKER` in the rendered entry -
`test_breaking_is_refused_on_a_compatible_bump`). `test_docs_verify.py`: four new tests mirroring
the existing migration-boundary tests but for the breaking-change marker. Full suite: 806 passed.
**`tools/CONTRACT.md` and `INSTALL.md`.** Both reworded to lead with the drop-in question and
treat migration as the second, independent one; `INSTALL.md` additionally tells the reader which
two lines to look for in release notes (`Breaking Change:` and `Migration:`) before applying an
update.
**`CHANGES.md`.** New `2.5.0` entry written after the bump, body filled in by the agent (the
tool leaves it empty by design) - includes the "what deliberately did not change" note that
`docs verify` still does not judge *whether* the chosen part was correct, matching the issue's
explicit scope-out.
## Turn 4 - publish, twice
`tools/wikitool version bump --minor --title "..."` was run first (11 changed files, drop-in in
both directions - the new requirement only binds the *next* boundary crossing, not
retroactively). `tools/wikitool publish --message "..."` then hit the **Mass-Update Gate**
(11 counted files ≥ threshold 10):
```
NEEDS USER CLEARANCE Mass-Update Gate: this publish would commit and push 11
counted files (>= threshold 10) to origin/main. ...
```
Per the gate's own instructions, the agent reproduced the full file-by-area breakdown and the
`--confirm <token>` line in its reply and ran nothing further that turn. The user replied
"freigegeben" for both the earlier publish authorization ("publish ist freigegeben") and, in a
separate turn, this specific token. The agent then ran `tools/wikitool publish --confirm
95ae372d5677 --message '...'`, which pushed commit `31662dc` to `origin/main`, and verified
`git rev-parse HEAD origin/main` matched afterward.
## Turn 5 - issue closeout and this capture
The user asked to update issue #26 "wie vorgeschlagen" (per the agent's own end-of-turn
suggestion), run `instructions/capture-session.md`, and in the same pass correct
`kb/entities/tools/wikitool.md`. The agent posted a comment on #26 summarizing what shipped
(including the parts that went beyond the issue's own checklist - the `--breaking` flag and the
second `docs verify` check, added because plain prose was judged likely to drift again) and
closed the issue. This transcript and the `kb/` correction are the remaining two steps of that
request.
## Outcome
| | |
|---|---|
| Version | `2.4.1``2.5.0` (`--minor`: new capability, still drop-in both directions) |
| Commit | `31662dc` on `main`, pushed to `origin` |
| Files changed | 11 (+432/-33): `instructions/dev/version-parts.md` (new), `instructions/dev/stack-dev/SKILL.md`, `tools/CONTRACT.md`, `tools/chemenu/version.py`, `tools/chemenu/commands/version_cmd.py`, `tools/chemenu/commands/docs_verify.py`, `tools/chemenu/tests/test_version_cmd.py`, `tools/chemenu/tests/test_docs_verify.py`, `CHANGES.md`, `INSTALL.md`, `VERSION` |
| Tests | 806 passed (`tools/chemenu/tests/`), including 6 new |
| Verification | `tools/wikitool docs verify` OK, `tools/wikitool instructions verify` OK (17 instructions, 6 skills, 12 published copies match), `tools/wikitool doctor` clean (only the expected `WIKITOOL_SESSION_ID` WARN) |
| Issues | #26 closed, comment `torben/chemenu#26` (issuecomment-474) |
| CI | Not yet observed in this session - a `VERSION` move on `main` triggers a tagged release per `.gitea/workflows/release.yml`; not polled |
+49 -21
View File
@@ -40,24 +40,25 @@ tools/wikitool <command> --help
| `touch --page "<Title>" [--summary "..."] [--provenance <v>] [--confidence-base <n>] [--date YYYY-MM-DD] [--set field=value ...] [--add field=value ...] [--remove field=value ...] [--no-date] [--dry-run]` | Update a page's own frontmatter: bump `modified:` and optionally rewrite any field its type declares. `--summary`/`--provenance`/`--confidence-base` are shorthands; `--set` reaches every other field and **replaces** its value, while `--add`/`--remove` change single elements of an array field (removing an absent element succeeds and says so). Repeating `--set` for one array field appends *within the call*, and `\,` is a literal comma - same rules as `new --set`. Refused with the command that owns them instead: `type:` (page-lifecycle), `confidence:` (derived - set `--confidence-base`), and the page-ref arrays `related:`/`sources:`/`entities:`/`concepts:` (`xref`). Everything else the schema declares is settable, and an unknown field lists what the page actually has. Schema-validates the fields it writes, and `raw_files:` entries must exist on disk. A source declares `date:` instead of `modified:`, and that is the *publication* date of the raw material - it is never bumped to today, and changes only when `--date` names a value explicitly. | | `touch --page "<Title>" [--summary "..."] [--provenance <v>] [--confidence-base <n>] [--date YYYY-MM-DD] [--set field=value ...] [--add field=value ...] [--remove field=value ...] [--no-date] [--dry-run]` | Update a page's own frontmatter: bump `modified:` and optionally rewrite any field its type declares. `--summary`/`--provenance`/`--confidence-base` are shorthands; `--set` reaches every other field and **replaces** its value, while `--add`/`--remove` change single elements of an array field (removing an absent element succeeds and says so). Repeating `--set` for one array field appends *within the call*, and `\,` is a literal comma - same rules as `new --set`. Refused with the command that owns them instead: `type:` (page-lifecycle), `confidence:` (derived - set `--confidence-base`), and the page-ref arrays `related:`/`sources:`/`entities:`/`concepts:` (`xref`). Everything else the schema declares is settable, and an unknown field lists what the page actually has. Schema-validates the fields it writes, and `raw_files:` entries must exist on disk. A source declares `date:` instead of `modified:`, and that is the *publication* date of the raw material - it is never bumped to today, and changes only when `--date` names a value explicitly. |
| `rename --from "<Old>" --to "<New>" [--dry-run]` | Rename a page and repoint every reference to it: body `[[wikilinks]]` (aliases and anchors preserved), a `[^cite-id]` whose id was derived from the old title (refreshed to match the new one, both in its Footnotes definition and every reference to it), the page's own H1, and every page-ref frontmatter array declared by the type's `page_ref_fields:`. If `--from` is *not* a page but is referenced, it instead repoints those references onto the existing `--to` page and moves nothing - the fix for a reference spelled `act_runner` when the page is `Act Runner` | | `rename --from "<Old>" --to "<New>" [--dry-run]` | Rename a page and repoint every reference to it: body `[[wikilinks]]` (aliases and anchors preserved), a `[^cite-id]` whose id was derived from the old title (refreshed to match the new one, both in its Footnotes definition and every reference to it), the page's own H1, and every page-ref frontmatter array declared by the type's `page_ref_fields:`. If `--from` is *not* a page but is referenced, it instead repoints those references onto the existing `--to` page and moves nothing - the fix for a reference spelled `act_runner` when the page is `Act Runner` |
| `rm --page "<Title>" [--yes] [--dry-run]` | Delete a page and mechanically de-link it. Refuses without `--yes` while other pages still reference it. Strips ref-array entries and bare `- [[Title]]` / `- **label:** [[Title]]` bullets; leaves prose and inline citations in place and reports them | | `rm --page "<Title>" [--yes] [--dry-run]` | Delete a page and mechanically de-link it. Refuses without `--yes` while other pages still reference it. Strips ref-array entries and bare `- [[Title]]` / `- **label:** [[Title]]` bullets; leaves prose and inline citations in place and reports them |
| `xref add --a "<A>" --b "<B>" --rel-a "<label>" --rel-b "<label>"` | Bidirectionally link two pages: frontmatter `related:` + body Relationships/See Also bullets. Idempotent. Refuses, before writing either side, when a page's type does not declare `related:` - a source page declares `entities:`/`concepts:` instead, and writing `related:` there produced frontmatter the schema rejects; the refusal names the fields the type does declare and points at `link-source`. | | `xref add --a "<A>" --b "<B>" --rel <label>` | Declare **one** edge: `A <label> B`, written into A's `related:` as `- <label>: B` and rendered into A's generated links region. B is not touched and does not point back - its inbound view is rendered from the graph. Idempotent, and re-running with a different label *relabels* rather than appending, since one page asserts one thing about another. Refuses before writing when the type does not declare `related:` (a source page declares `entities:`/`concepts:` - the refusal names them and points at `link-source`), and when `<label>` is not authorised by the source collection's `outbound:` block for the target's collection; that refusal lists the authorised set and points at `instructions/link-taxonomy.md` |
| `xref remove --a "<A>" --b "<B>" [--dry-run]` | Inverse of `xref add` *and* `xref link-source`: clears `<B>` from every page-ref frontmatter field `<A>`'s type declares (`related:`, `sources:`, `entities:`, `concepts:`) plus the matching bullets. It also sweeps a field the type does *not* declare but some other type does, and drops that key outright once empty - a leftover written before the check above existed has to stay repairable, or the page is a dead end. `--b` need not still exist as a page, so this is how a reference left by a hand-deleted or hand-renamed page gets cleared without hand-editing frontmatter. Idempotent. | | `xref remove --a "<A>" --b "<B>" [--dry-run]` | Clears the reference in **both** directions - it is the cleanup command for a deleted or hand-renamed page rather than the strict inverse of a one-directional `add`. Clears `<B>` from every page-ref frontmatter field `<A>`'s type declares (`related:`, `sources:`, `entities:`, `concepts:`) plus the matching bullets. It also sweeps a field the type does *not* declare but some other type does, and drops that key outright once empty - a leftover written before the check above existed has to stay repairable, or the page is a dead end. `--b` need not still exist as a page, so this is how a reference left by a hand-deleted or hand-renamed page gets cleared without hand-editing frontmatter. Idempotent. |
| `xref link-source --source "Source - X" --entities A,B,C` | Batch-link a source page to every entity/concept it mentions, **in both directions**: each target gets `sources:` + a See Also bullet, and the source page records each target in its own `entities:`/`concepts:`. Which of the two is chosen follows the target's collection (`kb/entities/` -> `entities:`), so a new collection needs no code change here. A target whose collection matches no reference field the source type declares is linked one-way and named in the output. Idempotent in both directions | | `xref link-source --source "Source - X" --entities A,B,C` | Batch-link a source page to every entity/concept it mentions: each target gets `sources:`, and the source page records each target in its own `entities:`/`concepts:`. No body bullet is written on either side - `sources:` *is* the record, and the See Also bullet this used to add was the reciprocal half of a model that no longer exists. Which of the two is chosen follows the target's collection (`kb/entities/` -> `entities:`), so a new collection needs no code change here. A target whose collection matches no reference field the source type declares is linked one-way and named in the output. Idempotent in both directions |
| `links show --page "<Title>" [--json]` | The declared graph around one page in both directions: the edges it asserts (from its own `related:`, with labels) and the edges other pages assert about it (computed across the corpus). The inbound half is derived rather than stored - that is what makes it complete, and it is the answer authored directional edges would otherwise have nowhere to come from. Read-only, exempt from the Iteration Budget Gate |
| `cite id --title "Source - X" [--file <qualifier>]` | Print the deterministic footnote id `cite add` would use for this (title, file) pair. Read-only, exempt from the Iteration Budget Gate | | `cite id --title "Source - X" [--file <qualifier>]` | Print the deterministic footnote id `cite add` would use for this (title, file) pair. Read-only, exempt from the Iteration Budget Gate |
| `cite add --page "<Title>" --source "Source - X" [--file <qualifier>] [--dry-run]` | Upsert a `[^cite-id]: [[Source - X]]` definition in the page's Footnotes block (reusing the id if the page already cites this exact source/file pair) and add `Source - X` to frontmatter `sources:`. Prints the `[^cite-id]` marker - pasting it into the prose is still a manual, editorial step | | `cite add --page "<Title>" --source "Source - X" [--file <qualifier>] [--dry-run]` | Upsert a `[^cite-id]: [[Source - X]]` definition in the page's generated footnotes region, creating it between `<!-- wikitool:footnotes -->` markers if absent (reusing the id if the page already cites this exact source/file pair) and add `Source - X` to frontmatter `sources:`. Prints the `[^cite-id]` marker - pasting it into the prose is still a manual, editorial step |
| `cite sync [--page "<Title>" \| --all] [--dry-run]` | Reconcile each page's Footnotes block against its actual `[^id]` references: prune definitions nothing references any more, re-render the block in first-reference order, and report any `[^id]` reference left with no definition | | `cite sync [--page "<Title>" \| --all] [--dry-run]` | Reconcile each page's footnotes region against its actual `[^id]` references: prune definitions nothing references any more, re-render the region in first-reference order, and report any `[^id]` reference left with no definition. A page still carrying the pre-4.0.0 undelimited block is converted to a marked region in the same pass - the marker carries the region's identity now, so re-rendering it under this instance's heading is a repair rather than a rename |
| `index rebuild [--dry-run]` | Regenerate the catalog from every page's frontmatter: `kb/index.md` becomes a map (statistics, one row per collection and per area, links to the shards) and the page tables are written to a generated `INDEX.md` in each collection. An area past 50 rows gets its own shard. Stale shards from removed collections/areas are deleted in the same pass | | `index rebuild [--dry-run]` | Regenerate the catalog from every page's frontmatter: `kb/index.md` becomes a map (statistics, one row per collection and per area, links to the shards) and the page tables are written to a generated `INDEX.md` in each collection. An area past 50 rows gets its own shard. Stale shards from removed collections/areas are deleted in the same pass |
| `log append --op ingest\|query\|lint\|create\|update\|delete\|rename --title "..." [--body "..."\|--body-file path]` | Append a formatted entry to `kb/log.md` | | `log append --op ingest\|query\|lint\|create\|update\|delete\|rename --title "..." [--body "..."\|--body-file path]` | Append a formatted entry to `kb/log.md` |
| `log status` | Read-only: count `ingest` entries logged since the last `lint` entry - the deterministic trigger behind the Maintenance Schedule's "every 10 sources" full-lint cadence | | `log status` | Read-only: count `ingest` entries logged since the last `lint` entry - the deterministic trigger behind the Maintenance Schedule's "every 10 sources" full-lint cadence |
| `lint [--json] [--markdown out.md] [--full] [--fail-on-error]` | Structural + provenance checks: broken wikilinks, dangling frontmatter references, orphan pages, index drift, schema gaps, duplicate titles, title mismatches, uncovered raw files, broken `raw_files:` refs, raw files claimed by more than one source page, unmarked provenance, citation/frontmatter drift, quote-limit overages (>2 blockquoted lines/page, advisory only). Prints only the sections that found something and always writes the full report to `reports/Lint Report <date>.md` (or `--markdown`), naming the path - `--full` prints everything, `--json` prints the findings and writes nothing | | `lint [--json] [--markdown out.md] [--full] [--fail-on-error]` | Structural + provenance checks: broken wikilinks, dangling frontmatter references, orphan pages, index drift, schema gaps, duplicate titles, title mismatches, uncovered raw files, broken `raw_files:` refs, raw files claimed by more than one source page, unmarked provenance, citation/frontmatter drift, quote-limit overages (>2 blockquoted lines/page, advisory only). Prints only the sections that found something and always writes the full report to `reports/Lint Report <date>.md` (or `--markdown`), naming the path - `--full` prints everything, `--json` prints the findings and writes nothing |
| `search ["<text>"] [--field <predicate> ...] [--kind/--subtype/--collection/--tag <v>] [--regex] [--limit N] [--sort [-]<field>] [--backend <name>] [--matches] [--json]` | Find pages in `kb/` without reading the index. Text search runs through a pluggable backend (`rg` today); `--field` predicates are evaluated on frontmatter - `f=v`, `f~substring`, `'f>=v'`, `'f:*'` (present), `'!f'` (absent), repeatable and ANDed. With no text this is a pure structured query. Results carry kind/summary/confidence so a hit can be judged without opening the page. Read-only, and **exempt from the Iteration Budget Gate** | | `search ["<text>"] [--field <predicate> ...] [--kind/--subtype/--collection/--tag <v>] [--regex] [--limit N] [--sort [-]<field>] [--backend <name>] [--matches] [--json]` | Find pages in `kb/` without reading the index. Text search runs through a pluggable backend (`rg` today); `--field` predicates are evaluated on frontmatter - `f=v`, `f~substring`, `'f>=v'`, `'f:*'` (present), `'!f'` (absent), repeatable and ANDed. With no text this is a pure structured query. Results carry kind/summary/confidence so a hit can be judged without opening the page. A page whose frontmatter does not parse can match no positive predicate, so it is **named** rather than dropped: `--json` always carries an `unreadable` list of `{path, reason}` (usually empty), and the table form writes the same lines to stderr. `--regex` is applied by `rg` alone, whose engine is linear; the ranking boosts for title and summary are literal-containment only, so a non-literal pattern is ranked by match count. `rg` is killed after 30 s and reported as a failure. Read-only, and **exempt from the Iteration Budget Gate** |
| `confidence decay [--apply]` | Recompute every page's derived `confidence` as `confidence_base * (1 - 0.01/month)`, floored at 0.2; dry-run by default | | `confidence decay [--apply]` | Recompute every page's derived `confidence` as `confidence_base * (1 - 0.01/month)`, floored at 0.2; dry-run by default |
| `confidence init-base [--apply]` | One-time backfill: set `confidence_base` from the current `confidence` on pages that predate the derived-confidence model | | `confidence init-base [--apply]` | One-time backfill: set `confidence_base` from the current `confidence` on pages that predate the derived-confidence model |
| `sources coverage [--json]` | List raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages | | `sources coverage [--json]` | List raw files with no source page, broken `raw_files:` references, and legacy directory/URL-only source pages |
| `sources trace --raw <path>` \| `--page "<Title>"` | Trace provenance in either direction: raw file -> source page(s) -> citing pages, or page -> its sources -> their raw files | | `sources trace --raw <path>` \| `--page "<Title>"` | Trace provenance in either direction: raw file -> source page(s) -> citing pages, or page -> its sources -> their raw files |
| `sources rebuild-index [--dry-run]` | Regenerate the `kb/provenance.md` reverse index (raw file -> source page -> citing pages) | | `sources rebuild-index [--dry-run]` | Regenerate the `kb/provenance.md` reverse index (raw file -> source page -> citing pages) |
| `sync [--remote origin] [--branch main] [--confirm-rebase TOKEN]` | Fetch `<remote>/<branch>` and bring the local branch up to date with it: fast-forward when the remote is simply ahead, rebase local commit(s) on top when both sides moved but touch disjoint files (a content conflict is then impossible by construction), and exit **42** for review when they touch the same file (the **rebase-review gate** - see `publish` below). Never commits, never pushes, never force-anything - no remote configured, or one that cannot be reached, is reported and skipped, not a failure. Meant to run once at the start of a writing session (`instructions/session-setup.md`) so the rest of it works against a current tree instead of discovering the drift at the final `publish` | | `sync [--remote origin] [--branch main] [--confirm-rebase TOKEN]` | Fetch `<remote>/<branch>` and bring the local branch up to date with it: fast-forward when the remote is simply ahead, rebase local commit(s) on top when both sides moved but touch disjoint files (a content conflict is then impossible by construction), and exit **42** for review when they touch the same file (the **rebase-review gate** - see `publish` below). Never commits, never pushes, never force-anything - no remote configured, or one that cannot be reached, is reported and skipped, not a failure. Meant to run once at the start of a writing session (`instructions/session-setup.md`) so the rest of it works against a current tree instead of discovering the drift at the final `publish` |
| `publish --message "<op>: <desc>" [--no-push] [--confirm TOKEN] [--confirm-rebase TOKEN] [--threshold N] [--remote origin] [--branch main] [--path P ...]` | Reconcile with `<remote>/<branch>` exactly like `sync` (skipped for `--no-push`), then stage all changes, commit, and push. If the reconcile step found a still-unpushed local commit and there is nothing new to stage, that commit is pushed anyway - a previous `publish` whose push failed no longer strands it. If the push is rejected despite the pre-check (a genuine race - something landed on the remote in between), one more reconcile-and-retry is attempted before giving up; never more than one. **Mass-Update Gate:** when >= `--threshold` (default 10) *counted* files would be committed, exits **42 (`EXIT_NEEDS_CLEARANCE`)** instead of publishing - a third outcome distinct from success (0) and a validation error (1) - and prints a review report: a scale line (file count, total lines added/removed, status breakdown), only-what-applies attention notes (deletions by name, control-plane and harness-config touches, published pages, the largest single change, binaries), and every counted path grouped by area with its status and churn, generated files split out as needing no review. The token digests each counted path **and its contents** plus the publish target, so a clearance carries neither to a different file list nor to edited contents; a wrong, invented or superseded token exits 42 again with the current state. Two kinds of path are committed but never counted and never shown for approval: anything under `work/`, and the files `wikitool` generates itself (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md`) - each is recomputable from the tree, so approving it decides nothing, and a routine ingest rebuilds five or six of them. The refusal line accounts for both, by reason. The gate is evaluated *before* anything is staged, so a refused publish leaves the working tree untouched. `--yes`/`-y` are gone and now fail with an explicit error. `--path` (repeatable) scopes the whole operation - gate count, staging, and commit - to a subtree | | `publish --message "<op>: <desc>" [--no-push] [--confirm TOKEN] [--confirm-rebase TOKEN] [--threshold N] [--remote origin] [--branch main] [--path P ...]` | Reconcile with `<remote>/<branch>` exactly like `sync` (skipped for `--no-push`), then stage all changes, commit, and push. If the reconcile step found a still-unpushed local commit and there is nothing new to stage, that commit is pushed anyway - a previous `publish` whose push failed no longer strands it. If the push is rejected despite the pre-check (a genuine race - something landed on the remote in between), one more reconcile-and-retry is attempted before giving up; never more than one. **Mass-Update Gate:** when >= `--threshold` (default 10) *counted* files would be committed, exits **42 (`EXIT_NEEDS_CLEARANCE`)** instead of publishing - a third outcome distinct from success (0) and a validation error (1) - and prints a review report: a scale line (file count, total lines added/removed, status breakdown), only-what-applies attention notes (deletions by name, control-plane and harness-config touches, published pages, the largest single change, binaries), and every counted path grouped by area with its status and churn, generated files split out as needing no review. The token digests each counted path **and its contents** plus the publish target, so a clearance carries neither to a different file list nor to edited contents; a wrong, invented or superseded token exits 42 again with the current state. Two kinds of path are committed but never counted and never shown for approval: anything under `work/`, and the files `wikitool` generates itself (`kb/index.md`, `kb/log.md`, `kb/provenance.md`, every `INDEX.md`) - each is recomputable from the tree, so approving it decides nothing, and a routine ingest rebuilds five or six of them. The refusal line accounts for both, by reason. The gate is evaluated *before* anything is staged, so a refused publish leaves the working tree untouched. **Publish-Remote Gate:** when this checkout carries a `.wikitool-remotes.json` and the resolved push URL of `--remote` is not listed in it, exits **42** before the reconcile step even fetches - the URL is read from `git remote get-url --push`, so a repointed remote does not pass on its name. Unlike the other two gates it has **no token and no flag**: the way past it is the user adding the URL to that file, and an agent editing it to get past a refusal is opening a gate on its own initiative. Absent file means unrestricted; a malformed one is an error, not permission. See [instructions/gates.md](../instructions/gates.md) `--yes`/`-y` are gone and now fail with an explicit error. `--path` (repeatable) scopes the whole operation - gate count, staging, and commit - to a subtree |
| `work new (--input <raw path> \| --key <run key>) [--again] [--dry-run]` | Scaffold `work/<runkey>/` for one workshop run: refuses a collision instead of suffixing it, and writes the required `README.md` + `plan.md`. `--input` derives the run key from the path below `raw/` (an ingest); `--key` names it outright for a run with no raw input - a migration or a sweep across `kb/` - and may not start with `ingest-`, which stays reserved for derived keys. Exactly one of the two. `--again` opens a dated second pass over a tree that has itself changed. See [work/CONTRACT.md](../work/CONTRACT.md) | | `work new (--input <raw path> \| --key <run key>) [--again] [--dry-run]` | Scaffold `work/<runkey>/` for one workshop run: refuses a collision instead of suffixing it, and writes the required `README.md` + `plan.md`. `--input` derives the run key from the path below `raw/` (an ingest); `--key` names it outright for a run with no raw input - a migration or a sweep across `kb/` - and may not start with `ingest-`, which stays reserved for derived keys. Exactly one of the two. `--again` opens a dated second pass over a tree that has itself changed. See [work/CONTRACT.md](../work/CONTRACT.md) |
| `work close --run-key <name> [--yes] [--dry-run]` | Delete a finished workshop. Lists what would be lost and requires `--yes`, because nothing in it is recoverable from the rest of the repo - the durable conclusions must already be in `kb/` | | `work close --run-key <name> [--yes] [--dry-run]` | Delete a finished workshop. Lists what would be lost and requires `--yes`, because nothing in it is recoverable from the rest of the repo - the durable conclusions must already be in `kb/` |
| `budget status` | Show the current session's `wikitool` call count and recent command history (never counted against the budget) | | `budget status` | Show the current session's `wikitool` call count and recent command history (never counted against the budget) |
@@ -67,25 +68,51 @@ tools/wikitool <command> --help
| `instructions sync [--force]` | Publish every `instructions/<name>/SKILL.md` into `.agents/skills/` and `.claude/skills/` as **copies**, and delete published skills whose source is gone. Both targets are gitignored, so a fresh clone runs this once - see `instructions/bootstrap.md`. Re-running is also how a drifted copy is repaired: the source always wins. `--force` is required only to replace a target directory that is not a published skill at all (no `SKILL.md` in it) | | `instructions sync [--force]` | Publish every `instructions/<name>/SKILL.md` into `.agents/skills/` and `.claude/skills/` as **copies**, and delete published skills whose source is gone. Both targets are gitignored, so a fresh clone runs this once - see `instructions/bootstrap.md`. Re-running is also how a drifted copy is repaired: the source always wins. `--force` is required only to replace a target directory that is not a published skill at all (no `SKILL.md` in it) |
| `instructions verify` | Check the instruction layer: flat instructions validate against `types/instruction.schema.yaml`, each `SKILL.md` carries the frontmatter its harness reads, every published copy is byte-identical to its source, no instruction is left that nothing references, and nothing under `instructions/dev/` is referenced from outside it (a `<!-- dist:strip-start/end -->` block is exempt - see [instructions/CONTRACT.md](../instructions/CONTRACT.md)). Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout | | `instructions verify` | Check the instruction layer: flat instructions validate against `types/instruction.schema.yaml`, each `SKILL.md` carries the frontmatter its harness reads, every published copy is byte-identical to its source, no instruction is left that nothing references, and nothing under `instructions/dev/` is referenced from outside it (a `<!-- dist:strip-start/end -->` block is exempt - see [instructions/CONTRACT.md](../instructions/CONTRACT.md)). Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout |
| `instructions list [--json]` | List the flat instructions with their descriptions. This is how the layer is discovered; `search` deliberately covers `kb/` only | | `instructions list [--json]` | List the flat instructions with their descriptions. This is how the layer is discovered; `search` deliberately covers `kb/` only |
| `docs verify` | Check the docs that mirror the code: every CLI command documented here (and vice versa), every directory under `kb/` has a `COLLECTION.md` and no directory outside it does, every stage contract present, no pre-migration `type: entity` blocks left in the contracts, and the `.gitignore` canaries clear in both directions (nothing ignored under `raw/`/`kb/`, everything ignored under `reports/` and the published skill directories) | | `docs verify` | Check the docs that mirror the code: every CLI command documented here (and vice versa), every directory under `kb/` has a `COLLECTION.md` and no directory outside it does, every collection declaring `profile:` and a `required_by_stack:` that agrees with the stack's own list, `kb/CONVENTIONS.md` naming all three tool-owned section headings if it exists at all, every stage contract present, no pre-migration `type: entity` blocks left in the contracts, and the `.gitignore` canaries clear in both directions (nothing ignored under `raw/`/`kb/`, everything ignored under `reports/` and the published skill directories) |
| `eval sessions [--json]` | List the sessions that have a trace under `reports/telemetry/`, most recent first. Read-only and exempt from the Iteration Budget Gate | | `eval sessions [--json]` | List the sessions that have a trace under `reports/telemetry/`, most recent first. Read-only and exempt from the Iteration Budget Gate |
| `eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]` | Score one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged. Defaults to the current session. `--save` writes `reports/evals/<date>/<session>.{json,md}`. Read-only over `kb/` and exempt from the budget; see [../EVALS.md](../EVALS.md) | | `eval score [--session <id>] [--json] [--markdown out.md] [--save] [--fail-on-error]` | Score one traced session: structural state from `lint`'s own checks (L1) plus trajectory rules over the trace (L2) - was a refused call repeated unchanged, was a gate flag passed without that gate having refused anything, did a publish of `kb/` pages go unlogged. Defaults to the current session. `--save` writes `reports/evals/<date>/<session>.{json,md}`. Read-only over `kb/` and exempt from the budget; see [../EVALS.md](../EVALS.md) |
| `dist export <target> [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]` | Write a contentless, distributable copy of this repo's machinery into an empty `<target>` directory: `AGENTS.md`/`README.md`/`EVALS.md` with any `<!-- dist:strip-start -->...<!-- dist:strip-end -->` region removed, `instructions/` (minus `instructions/dev/`), `types/`, `tools/` (no venv/caches), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, every `kb/*/COLLECTION.md` (no pages, no areas), empty `raw/{articles,documents,notes,assets}/`, `VERSION`, `USER.md.template`/`SOUL.md.template` (the templates ship; a filled `USER.md`/`SOUL.md` never does - the root allowlist is what makes that automatic), and a generated `.wikitool-release.json` stamp (version, export date, origin, and a sha256 per exported file - the base a later upgrade would compare against). The four origin options only fill stamp fields: `export` never calls git and cannot discover them. Refuses a non-empty target, and a tree with no `VERSION`. See `instructions/setup-instance.md`. One-way: there is no command that reconstructs a distributed instance into a dev instance - work on the stack in the origin repo (or a new dev instance exported from it) instead | | `dist export <target> [--dry-run] [--source-repo U] [--source-commit SHA] [--release-url U] [--update-url U]` | Write a contentless, distributable copy of this repo's machinery into an empty `<target>` directory: `AGENTS.md`/`README.md`/`EVALS.md` with any `<!-- dist:strip-start -->...<!-- dist:strip-end -->` region removed, `instructions/` (minus `instructions/dev/`), `types/` (the `root: kb` page type-specs and their schemas re-keyed as `.template`, the stack's own verbatim), `tools/` (no venv/caches), the `.github/hooks/`+`.vibe/` session-tracing config plus `.claude/settings.json`, `kb/CONTRACT.md` (no pages, no areas), empty `raw/{articles,documents,notes,assets}/`, `VERSION`, `USER.md.template`/`SOUL.md.template` plus `kb/CONVENTIONS.md.template` and each collection's contract re-keyed as `kb/<name>/COLLECTION.md.template` (the templates ship; the filled `USER.md`/`SOUL.md`/`kb/CONVENTIONS.md`/`kb/<name>/COLLECTION.md`/`types/<page-type>.md` never do - all of them bind their instance and none are the stack's to decide, and `find_leaks` refuses a plan carrying one), and a generated `.wikitool-release.json` stamp (version, export date, origin, and a sha256 per exported file - the base a later upgrade would compare against). The four origin options only fill stamp fields: `export` never calls git and cannot discover them. Refuses a non-empty target, and a tree with no `VERSION`. See `instructions/setup-instance.md`. One-way: there is no command that reconstructs a distributed instance into a dev instance - work on the stack in the origin repo (or a new dev instance exported from it) instead |
| `version show [--json]` | Print this instance's stack version and where it came from (development tree, or a distribution with its export date and origin). Bare `wikitool version` is an alias for this. Read-only, offline, and **exempt from the Iteration Budget Gate** | | `version show [--json]` | Print this instance's stack version and where it came from (development tree, or a distribution with its export date and origin). Bare `wikitool version` is an alias for this. Read-only, offline, and **exempt from the Iteration Budget Gate** |
| `version check [--url U] [--timeout S] [--json]` | Ask the origin's release feed whether a newer stack exists, and whether the step crosses a compatibility boundary (`state: current\|update\|migration\|ahead`). **The only command in `wikitool` that makes a network call** - never reached implicitly from another command, needs no key, times out, and reports an unreachable feed as an error rather than as "up to date". The feed is `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin; `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously. Read-only and exempt from the budget gate | | `version check [--url U] [--timeout S] [--json]` | Ask the origin's release feed whether a newer stack exists, and whether the step crosses a compatibility boundary (`state: current\|update\|migration\|ahead`). **The only command in `wikitool` that makes a network call** - never reached implicitly from another command, needs no key, times out, and reports an unreachable feed as an error rather than as "up to date". The feed is `$WIKITOOL_UPDATE_URL`, else the release stamp's, else the built-in origin; `$WIKITOOL_UPDATE_TOKEN` is only needed if that feed is not readable anonymously. Read-only and exempt from the budget gate |
| `version notes [--version X.Y.Z]` | Print one version's `CHANGES.md` entry, for use as release notes (default: this tree's `VERSION`). Read-only and exempt from the budget gate | | `version notes [--version X.Y.Z]` | Print one version's `CHANGES.md` entry, for use as release notes (default: this tree's `VERSION`). Read-only and exempt from the budget gate |
| `version bump --major\|--minor\|--patch --title "<...>" [--no-migration "<reason>"] [--dry-run]` | Raise `VERSION` and open the matching `CHANGES.md` entry - heading, date and author only; the body stays the author's to write, the way `new` writes frontmatter and leaves the prose. Refuses more or fewer than one part, an empty title, and a changelog already documenting a version that is not older than the new one. Compatibility follows the **leftmost non-zero component**, which for this stack (at `1.0.0` and up, no pre-release suffixes anywhere) means MAJOR: PATCH is a fix, MINOR a compatible capability, MAJOR requires instances to migrate. A MAJOR bump additionally requires a migration document targeting the new version, or `--no-migration "<reason>"`, which records the reason in the entry | | `version bump --major\|--minor\|--patch --title "<...>" [--breaking "<what breaks>"] [--no-migration "<reason>"] [--dry-run]` | Raise `VERSION` and open the matching `CHANGES.md` entry - heading, date and author only; the body stays the author's to write, the way `new` writes frontmatter and leaves the prose. Refuses more or fewer than one part, an empty title, and a changelog already documenting a version that is not older than the new one. Compatibility follows the **leftmost non-zero component**, which for this stack (at `1.0.0` and up, no pre-release suffixes anywhere) means MAJOR: PATCH is a fix, MINOR a compatible capability, MAJOR a version that is **not a drop-in replacement** - any hand-work on update, or a downgrade that no longer works. Whether content must be migrated is a second, independent question. A MAJOR bump therefore requires `--breaking "<what stops working>"`, which is refused on any other part, and on top of it a migration document targeting the new version or `--no-migration "<reason>"`; both are recorded in the entry. Which part a change earns stays a judgment call: the command enforces that a crossing documents itself, never that the part was chosen correctly |
| `migrate list [--json]` | List every migration document under `instructions/migrations/`, oldest target first, with its kind. Read-only and **exempt from the Iteration Budget Gate** | | `migrate list [--json]` | List every migration document under `instructions/migrations/`, oldest target first, with its kind and obligation. Read-only and **exempt from the Iteration Budget Gate** |
| `migrate status [--json]` | Show the migrations this instance still owes, in the order they must run: every document whose `migrates_to` lies in `(kb_version, VERSION]`. Exits 1 only when `.wikitool-kb.json` is missing - the content's shape is a question the tool refuses to answer by guessing. Read-only and exempt from the budget gate | | `migrate status [--json]` | Show the migrations this instance still owes, in the order they must run: every **required** document whose `migrates_to` lies in `(kb_version, VERSION]`. `offered` documents are listed separately above the chain and never block, never count as owed, and are bounded by the applied ledger rather than by `kb_version` - taking one deliberately does not move the version, so the version cannot say whether it was taken. When a release stamp is present, also reports which shipped files this instance has since edited (from the per-file sha256 in `.wikitool-release.json`), which is what says whether an offer may be copied over or has to be reconciled by hand; without a stamp that question is reported as unanswerable rather than answered. Exits 1 only when `.wikitool-kb.json` is missing - the content's shape is a question the tool refuses to answer by guessing. Read-only and exempt from the budget gate |
| `migrate verify --from <rev> [--path P ...] [--expect-body-change] [--json] [--fail-on-error]` | Compare `kb/` against a git revision on the invariants a content migration must not change: wikilink and citation **counts** (not sets), footnote definitions, H1, and structural frontmatter. Reports added/removed pages without failing on them. `--expect-body-change` additionally flags a page whose body did not change at all. Not migration-specific - worth running after any bulk rewrite, and the one question `lint` cannot answer, since it reads a single revision and so cannot see that something went missing. Read-only and exempt from the budget gate | | `migrate verify --from <rev> [--path P ...] [--expect-body-change] [--json] [--fail-on-error]` | Compare `kb/` against a git revision on the invariants a content migration must not change: wikilink and citation **counts** (not sets), footnote definitions, H1, structural frontmatter, and the **count of generated-region marker pairs** - a page that went from one links region to two has the same set of region names and a different count, and a lost marker turns a generated region into prose the next write appends a second one beside. Reports added/removed pages without failing on them. `--expect-body-change` additionally flags a page whose body did not change at all. Not migration-specific - worth running after any bulk rewrite, and the one question `lint` cannot answer, since it reads a single revision and so cannot see that something went missing. Read-only and exempt from the budget gate |
| `migrate done <version> [--pages N] [--dry-run]` | Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json` to its target. **Refuses any version that is not the next link in the chain** - skipping one leaves the corpus in a shape no version describes, and an interrupted multi-step upgrade has to be resumable rather than guessable | | `migrate done <version> [--pages N] [--dry-run]` | Record one migration as applied, advancing `kb_version` in `.wikitool-kb.json` to its target. **Refuses any version that is not the next link in the chain** - skipping one leaves the corpus in a shape no version describes, and an interrupted multi-step upgrade has to be resumable rather than guessable. An `offered` migration is recorded in the applied ledger *without* moving `kb_version` and with no ordering rule applied: it is not a link in the chain, so there is nothing to skip, and requiring the chain first would make an unrelated file upgrade wait on it. Re-recording one already in the ledger is a no-op, not an error |
| `migrate baseline <version> [--force]` | Declare `kb_version` once, for an instance predating `.wikitool-kb.json`. Refuses to overwrite an existing declaration without `--force`: advancing after a migration is `done`, which checks the chain, and this command must not become the quiet way around it | | `migrate baseline <version> [--force]` | Declare `kb_version` once, for an instance predating `.wikitool-kb.json`. Refuses to overwrite an existing declaration without `--force`: advancing after a migration is `done`, which checks the chain, and this command must not become the quiet way around it |
| `doctor [--json]` | Check that this instance is correctly configured: dependencies (Python, ripgrep), author resolution, stack version, git identity/branch/remote, published skills, kb/raw/reports/work/instructions structure, personalization (`USER.md`/`SOUL.md` present **and** filled - a file still carrying the template's sentinel is a `FAIL`, since a renamed template is not a filled one), the environment note (`ENVIRONMENT.md` - optional, so absent is `OK`; a still-templated one is a `WARN`), generated files, and `WIKITOOL_SESSION_ID`. Read-only, exit 1 only on a `FAIL` (a missing remote, session id, or `VERSION` is a `WARN`, not a fault). Exempt from the Iteration Budget Gate | | `doctor [--json]` | Check that this instance is correctly configured: dependencies (Python, ripgrep), author resolution, stack version, git identity/branch/remote, published skills, kb/raw/reports/work/instructions structure, personalization (`USER.md`/`SOUL.md` present **and** filled - a file still carrying the template's sentinel is a `FAIL`, since a renamed template is not a filled one), the KB conventions (`kb/CONVENTIONS.md` present, unsentinelled, and naming all three tool-owned section headings - a `FAIL` on any of the three, because `xref`/`cite` write out of it), the environment note (`ENVIRONMENT.md` - optional, so absent is `OK`; a still-templated one is a `WARN`), generated files, and `WIKITOOL_SESSION_ID`. Read-only, exit 1 only on a `FAIL` (a missing remote, session id, or `VERSION` is a `WARN`, not a fault). Exempt from the Iteration Budget Gate |
## Design notes ## Design notes
- All commands operate on the real repo (paths resolved relative to this - All commands operate on the real repo, so they can be run from any working
file's location), so they can be run from any working directory. directory. The root is resolved by precedence - an explicit argument, then
`$CHEMENU_ROOT`, then a walk up from the package's own location - and the
walk-up is the default, so `tools/wikitool` with no variable set behaves
exactly as it always has. Nothing under the root is bound at import time:
`KB_DIR`, `RAW_DIR` and the rest follow whatever `ROOT` currently is, which
is what makes the half-repointed state (a moved `ROOT` with a stale `KB_DIR`)
unconstructible rather than merely discouraged.
- **The library boundary.** `chemenu.api.Corpus` is the in-process entry point:
it takes a corpus root, returns the same structures the `--json` forms print,
and raises `ChemenuError` where the CLI prints `ERROR` and exits 1. It is
read-only *structurally* - nothing under `chemenu.commands` is imported from
it, so `new`, `publish` and the rest are not reachable, rather than filtered.
The cores it calls (`search/service.py`, `lint_core.py`, `types_core.py`)
import no `typer` and no `rich`; the modules under `commands/` are the
terminal adapters over them. A second consumer is therefore a second adapter,
not a second implementation.
- **The MCP read server** (`chemenu/mcp/`) is that second adapter: `search`,
`types`, `describe_type`, `lint` and `status` over `chemenu.api`, on `stdio`
or `streamable-http`. Its dependency is optional and lives in
`requirements-mcp.txt`, so a CLI-only instance does not install it. There is
no write tool - nothing under `commands/` is importable from it, so the write
functions are unreachable rather than filtered - and every response carries
the commit it was computed from. Running it, and keeping its checkout
current, is [instructions/mcp-read-server.md](../instructions/mcp-read-server.md).
Authentication and rate limiting are middleware in front of the process, not
code here; the Iteration Budget Gate is deliberately not applied, because it
bounds an agent session rather than a user.
- `new`/`xref`/`log append` only produce structurally-correct frontmatter and - `new`/`xref`/`log append` only produce structurally-correct frontmatter and
body skeletons/edits - the prose (Description, Summary, judgment calls body skeletons/edits - the prose (Description, Summary, judgment calls
about relationships) is still written by the LLM afterwards. about relationships) is still written by the LLM afterwards.
@@ -151,10 +178,10 @@ is atomic, and whether a retry is safe.
| `index rebuild` / `sources rebuild-index` | Rare I/O error only | Yes - the file is regenerated from scratch | Safe to retry freely || `log append` | Invalid `--op` or unreadable `--body-file` | Yes - single append | **Not idempotent.** If the previous run's outcome is uncertain, check the tail of `kb/log.md` before retrying | | `index rebuild` / `sources rebuild-index` | Rare I/O error only | Yes - the file is regenerated from scratch | Safe to retry freely || `log append` | Invalid `--op` or unreadable `--body-file` | Yes - single append | **Not idempotent.** If the previous run's outcome is uncertain, check the tail of `kb/log.md` before retrying |
| `log status` | Never fails (reports 0 if `kb/log.md` is missing or empty) | Read-only | Safe to retry freely | | `log status` | Never fails (reports 0 if `kb/log.md` is missing or empty) | Read-only | Safe to retry freely |
| `lint` | Only with `--fail-on-error`: hard findings exist | Writes one report file (single atomic write) unless `--json` | Safe to retry freely, but re-run it to re-*measure*, never to re-read: the printed path holds the full report. Exit 1 means "act on the findings", not "the tool is broken" | | `lint` | Only with `--fail-on-error`: hard findings exist | Writes one report file (single atomic write) unless `--json` | Safe to retry freely, but re-run it to re-*measure*, never to re-read: the printed path holds the full report. Exit 1 means "act on the findings", not "the tool is broken" |
| `search` | `rg` is not installed, a malformed `--field` predicate, an unknown field name, or an unknown `--backend` | Read-only | Fix the argument and retry. An unknown field name is reported with the list of fields that do exist - it is never answered with an empty result, because that would read as "no such pages" | | `search` | `rg` is not installed or did not finish within 30 s, a malformed `--field` predicate, an unknown field name, or an unknown `--backend` | Read-only | Fix the argument and retry. A timeout is a pathological pattern or an unresponsive corpus directory, not a slow answer - narrow the query or drop `--regex` rather than retrying it unchanged. An unknown field name is reported with the list of fields that do exist - it is never answered with an empty result, because that would read as "no such pages" |
| `confidence decay --apply` / `init-base --apply` | Rare I/O error mid-loop | No - one write per page | Safe to retry freely; both recompute from `confidence_base` and never compound | | `confidence decay --apply` / `init-base --apply` | Rare I/O error mid-loop | No - one write per page | Safe to retry freely; both recompute from `confidence_base` and never compound |
| `sync` | The automatic rebase hit a real conflict (git failed) | No - fetch, then at most one merge/rebase attempt, aborted cleanly on failure | For a conflict: **do not retry, do not force** - resolve manually and re-run. **Exit 42, not 1**, when the rebase-review gate needs clearance: show the user the command's full output verbatim (upstream commits, the overlapping files, their diff) and stop; re-running with `--confirm-rebase <token>` clears it, and a wrong, invented, or superseded token exits 42 again with the current state. No remote configured, or one that cannot be reached, is not a failure - reported and skipped | | `sync` | The automatic rebase hit a real conflict (git failed) | No - fetch, then at most one merge/rebase attempt, aborted cleanly on failure | For a conflict: **do not retry, do not force** - resolve manually and re-run. **Exit 42, not 1**, when the rebase-review gate needs clearance: show the user the command's full output verbatim (upstream commits, the overlapping files, their diff) and stop; re-running with `--confirm-rebase <token>` clears it, and a wrong, invented, or superseded token exits 42 again with the current state. No remote configured, or one that cannot be reached, is not a failure - reported and skipped |
| `publish` | git failed, **or** `--yes`/`-y` was passed. **Exit 42, not 1**, when the Mass-Update Gate or the rebase-review gate (raised by the same reconcile `sync` performs) needs clearance | No - sequential git operations, but both gates run before staging | For git failures: **do not retry, do not force** - report and ask the user (the reconcile step already retried the push once on its own, if a rebase resolved the rejection). For exit 42: show the user the command's full output verbatim and stop; it names the evidence and the `--confirm <token>` or `--confirm-rebase <token>` line to re-run, and re-running without it exits 42 again | | `publish` | git failed, **or** `--yes`/`-y` was passed. **Exit 42, not 1**, when the Mass-Update Gate, the rebase-review gate (raised by the same reconcile `sync` performs), or the Publish-Remote Gate refuses | No - sequential git operations, but both gates run before staging | For git failures: **do not retry, do not force** - report and ask the user (the reconcile step already retried the push once on its own, if a rebase resolved the rejection). For exit 42: show the user the command's full output verbatim and stop; it names the evidence and the `--confirm <token>` or `--confirm-rebase <token>` line to re-run, and re-running without it exits 42 again. The Publish-Remote Gate is the exception with no such line: it names the push URL that would have been written to and the ones this checkout allows, and only the user resolves it |
| `work new` | Neither or both of `--input`/`--key` given, `--input` outside `raw/`, a `--key` that is empty or starts with `ingest-`, or the workshop already exists | Yes - one directory with two files | A collision is not transient: resume the existing run instead, or pass `--again` if the tree itself changed. Never create a numbered variant by hand | | `work new` | Neither or both of `--input`/`--key` given, `--input` outside `raw/`, a `--key` that is empty or starts with `ingest-`, or the workshop already exists | Yes - one directory with two files | A collision is not transient: resume the existing run instead, or pass `--again` if the tree itself changed. Never create a numbered variant by hand |
| `work close` | Unknown run key, or `--yes` was not passed | No - a recursive delete | For "not confirmed": check the listed files are no longer needed, confirm the conclusions are in `kb/`, then re-run with `--yes` | | `work close` | Unknown run key, or `--yes` was not passed | No - a recursive delete | For "not confirmed": check the listed files are no longer needed, confirm the conclusions are in `kb/`, then re-run with `--yes` |
| `sources coverage` / `sources trace` | Bad arguments (e.g. neither or both of `--raw`/`--page`) | Read-only | Fix the argument and retry | | `sources coverage` / `sources trace` | Bad arguments (e.g. neither or both of `--raw`/`--page`) | Read-only | Fix the argument and retry |
@@ -164,10 +191,11 @@ is atomic, and whether a retry is safe.
| `dist export` | Target exists and is not empty, is not a directory, or the tree has no readable `VERSION` | Yes - nothing is written until every file is planned | Point `<target>` at an empty (or new) directory and retry. Never merge into a non-empty one by hand | | `dist export` | Target exists and is not empty, is not a directory, or the tree has no readable `VERSION` | Yes - nothing is written until every file is planned | Point `<target>` at an empty (or new) directory and retry. Never merge into a non-empty one by hand |
| `version show` / `version notes` | `VERSION` is missing or unparseable; for `notes`, no `CHANGES.md` entry names the version asked for | Read-only | Fix `VERSION`, or write the changelog entry (`version bump` writes its heading). Safe to retry | | `version show` / `version notes` | `VERSION` is missing or unparseable; for `notes`, no `CHANGES.md` entry names the version asked for | Read-only | Fix `VERSION`, or write the changelog entry (`version bump` writes its heading). Safe to retry |
| `version check` | The feed could not be reached, answered non-JSON, or carried no `tag_name`. **Never** answers "up to date" for a question it could not ask | Read-only, no local writes | A network failure is transient - retry once, then report it. HTTP 401/403 names `$WIKITOOL_UPDATE_TOKEN`; 404 means no release exists yet or the URL points at the wrong repo | | `version check` | The feed could not be reached, answered non-JSON, or carried no `tag_name`. **Never** answers "up to date" for a question it could not ask | Read-only, no local writes | A network failure is transient - retry once, then report it. HTTP 401/403 names `$WIKITOOL_UPDATE_TOKEN`; 404 means no release exists yet or the URL points at the wrong repo |
| `version bump` | More or fewer than one of `--major/--minor/--patch`, an empty `--title`, a missing `VERSION`/`CHANGES.md`, a changelog already documenting a version not older than the new one, a boundary-crossing bump with neither a migration document nor `--no-migration`, or `--no-migration` on a bump that crosses nothing | No - `VERSION` then `CHANGES.md` | **Not idempotent**: a second run bumps again. If the outcome is uncertain, read `VERSION` and the top of `CHANGES.md` before retrying | | `version bump` | More or fewer than one of `--major/--minor/--patch`, an empty `--title`, a missing `VERSION`/`CHANGES.md`, a changelog already documenting a version not older than the new one, a boundary-crossing bump without `--breaking` or with neither a migration document nor `--no-migration`, or `--breaking`/`--no-migration` on a bump that crosses nothing | No - `VERSION` then `CHANGES.md` | **Not idempotent**: a second run bumps again. If the outcome is uncertain, read `VERSION` and the top of `CHANGES.md` before retrying |
| `links show` | Page not found | Read-only | Check the exact title with `search`; a wikilink target is not always the page's stem |
| `migrate list` / `migrate status` | `list` never fails; `status` exits 1 when `.wikitool-kb.json` is missing or unreadable, or `VERSION` is | Read-only | For a missing declaration: run `migrate baseline <version>` once, then retry. Safe to retry freely otherwise | | `migrate list` / `migrate status` | `list` never fails; `status` exits 1 when `.wikitool-kb.json` is missing or unreadable, or `VERSION` is | Read-only | For a missing declaration: run `migrate baseline <version>` once, then retry. Safe to retry freely otherwise |
| `migrate verify` | Only with `--fail-on-error`: an invariant changed. Also exits 1 if `--from` is not a revision in this repository | Read-only | Exit 1 from `--fail-on-error` means "act on the findings", not "the tool is broken". A finding is never fixed by re-running - it names a page and what changed on it | | `migrate verify` | Only with `--fail-on-error`: an invariant changed. Also exits 1 if `--from` is not a revision in this repository | Read-only | Exit 1 from `--fail-on-error` means "act on the findings", not "the tool is broken". A finding is never fixed by re-running - it names a page and what changed on it |
| `migrate done` | Unknown version, no `.wikitool-kb.json`, nothing outstanding, or a version that is not the next link in the chain | Yes - single file write | **Not idempotent**: it advances the chain. For "not the next link", run `migrate status` and apply them in the order it prints - never force the order | | `migrate done` | Unknown version, no `.wikitool-kb.json`, nothing outstanding, or a *required* version that is not the next link in the chain | Yes - single file write | **Not idempotent** for a required migration: it advances the chain. For "not the next link", run `migrate status` and apply them in the order it prints - never force the order. Recording an `offered` migration *is* idempotent and safe to repeat |
| `migrate baseline` | Unparseable version, or a declaration already exists and `--force` was not passed | Yes - single file write | Safe to re-run with the same version. If a declaration exists, it is almost always `migrate done` that was wanted | | `migrate baseline` | Unparseable version, or a declaration already exists and `--force` was not passed | Yes - single file write | Safe to re-run with the same version. If a declaration exists, it is almost always `migrate done` that was wanted |
| `doctor` | At least one check reported `FAIL` (a `WARN`, e.g. no remote or no `WIKITOOL_SESSION_ID`, does not exit 1) | Read-only | Each finding names its own fix command; re-run after applying it | | `doctor` | At least one check reported `FAIL` (a `WARN`, e.g. no remote or no `WIKITOOL_SESSION_ID`, does not exit 1) | Read-only | Each finding names its own fix command; re-run after applying it |
| `budget status` / `budget reset` | `reset` without `--yes`; `status` never fails | Read/rewrite of one JSON file | `status` is safe to retry. For `reset`: get the user's approval, then re-run with `--yes` | | `budget status` / `budget reset` | `reset` without `--yes`; `status` never fails | Read/rewrite of one JSON file | `status` is safe to retry. For `reset`: get the user's approval, then re-run with `--yes` |
+50 -14
View File
@@ -34,20 +34,42 @@ tools/
wikitool entry point wikitool entry point
chemenu/ chemenu/
cli.py Typer app: registers every command, runs the budget gate cli.py Typer app: registers every command, runs the budget gate
config.py repo layout constants (ROOT, RAW_DIR, KB_DIR, WORK_DIR, ...) config.py repo layout: root resolution and every path under it
api.py the in-process entry point - point Chemenu at a corpus and read it
errors.py ChemenuError / ValidationError / BackendError
corpus_cache.py one parsed corpus per commit, never cached while the tree is dirty
kb_scan.py page iteration/loading over kb/ kb_scan.py page iteration/loading over kb/
kb_collections.py collection discovery (a directory with COLLECTION.md) blocks.py generated regions in a page body, found by marker rather than by heading
links.py labelled edges in `related:` - the graph's semantics as data, not prose
kb_collections.py collection discovery (a directory with COLLECTION.md), and what one declares about itself
conventions.py kb/CONVENTIONS.md: what this instance decided about authoring, as opposed to what the stack enforces
type_resolver.py type-spec loading and schema resolution type_resolver.py type-spec loading and schema resolution
sections.py the section headings the tool reads and writes in a page body lint_core.py the lint checks and the report, with no CLI attached
types_core.py type-spec listing/description, with no CLI attached
markdown_code.py masks code spans/fences so a page may show wiki notation, not only use it markdown_code.py masks code spans/fences so a page may show wiki notation, not only use it
version.py the stack version: VERSION, the release stamp, the compatibility rule version.py the stack version: VERSION, the release stamp, the compatibility rule
kb_state.py the KB version (.wikitool-kb.json) and the migration chain kb_state.py the KB version (.wikitool-kb.json) and the migration chain
corpus_diff.py invariant comparison of kb/ between two revisions corpus_diff.py invariant comparison of kb/ between two revisions
search/ pluggable search backends (base protocol, ripgrep, filters, fuse) search/ pluggable search backends, plus service.py - the search core
commands/ one module per command or command group commands/ one module per command or command group: the terminal adapters
tests/ pytest suite tests/ pytest suite
``` ```
**Two consumers, one core.** The CLI is not the only caller any more. The cores
(`search/service.py`, `lint_core.py`, `types_core.py`) hold what decides an
answer and import no `typer` and no `rich`; the modules under `commands/` turn
those values into terminal output and those exceptions into exit codes.
`api.Corpus` is the in-process entry point over the same functions - it takes a
corpus root, returns exactly the structures the `--json` forms print, and
raises instead of exiting. A second consumer is therefore a second adapter
rather than a second implementation, and the write commands are unreachable
from `api` because nothing under `commands/` is imported there.
**Which corpus.** The root resolves by precedence: an explicit argument, then
`$CHEMENU_ROOT`, then a walk up from the package's own location. The walk-up is
the default, so the CLI is unaffected by any of this. Nothing under the root is
bound at import time - `KB_DIR` and friends follow whatever `ROOT` currently is.
## Adding a command ## Adding a command
1. Write the module under `chemenu/commands/`. A group is a `typer.Typer()` 1. Write the module under `chemenu/commands/`. A group is a `typer.Typer()`
@@ -90,15 +112,29 @@ procedure written down in advance is one an agent can complete alone. Whether
a human *actually* saw it is not enforced here - that question is answered in a human *actually* saw it is not enforced here - that question is answered in
the eval layer (`evals/trajectory.py`, `clearance-ended-the-turn`). the eval layer (`evals/trajectory.py`, `clearance-ended-the-turn`).
**Section names are a vocabulary, not literals.** `xref add` writes into Relationships and See **Nothing locates a region by its prose.** `xref` owns the links region and `cite` the footnotes
Also, and `cite add` owns the trailing Footnotes block, so those three headings are structure the region, and each is delimited by a `<!-- wikitool:<name> -->` marker pair (`blocks.py`). The
tool matches on. They are named once in `sections.py`, and each has one canonical spelling - what heading inside is rendered from `kb/CONVENTIONS.md` and is replaced along with the rest of the
the tool writes - plus aliases it still recognizes. That asymmetry is what let the wiki be region on every write - so no heading text exists in Python, and changing the declaration cannot
translated page by page instead of atomically: an untranslated `## Relationships` is still found split a page.
and appended to. Dropping an alias is therefore a breaking change for any page not yet converted,
not a cleanup. Renaming a heading is a migration's job; no other command may do it as a side Both halves of that mattered. Matching on the heading made the KB language a compiler constant;
effect (see `cite_block_heading` in `provenance.py`, which exists solely so `cite sync` stays a *guessing* where the region ended - at the next heading, and before that at the end of the file -
no-op on an untranslated page). silently deleted content sitting after it on eight pages. `migrate verify` compares marker-pair
counts for the same reason it compares wikilink counts: a dropped marker is invisible otherwise.
**A relationship label is data, not prose.** `related:` carries `- <label>: <target>`
(`links.py`), the label drawn from `instructions/link-taxonomy.md` and authorised per
destination by the *source* collection's `outbound:` block. The body bullet is a rendering of
that, which is what removed the need to parse a German phrase back into a relationship - and why
the vocabulary can be checked at all, after drifting to 152 distinct labels while it could not
be. Both readers accept a bare title as an unlabelled edge: that is the shape a page is in
between the machinery landing and the migration reaching it, and `lint` is what reports it.
**An edge is authored in one direction.** `xref add` writes one, on the asserting page. The
inbound view is rendered from the graph rather than stored, so navigation does not depend on
anyone writing a mirror - and per-collection authorisation stays coherent, which it cannot be if
the tool writes edges into a collection whose rules the author never read.
**Generated output is never committed.** `reports/`, `.agents/skills/` and **Generated output is never committed.** `reports/`, `.agents/skills/` and
`.claude/skills/` are build output; `docs verify` carries canaries in both `.claude/skills/` are build output; `docs verify` carries canaries in both
+202
View File
@@ -0,0 +1,202 @@
"""The in-process entry point: point Chemenu at a corpus and read from it.
This is the seam the MCP server (Gitea #19) is built on, and the reason it is
worth having as its own module rather than as "call the command functions
yourself": it fixes the two things that made an in-process caller a second-class
one.
**A corpus you name, not the one this package happens to sit in.** `Corpus`
holds the root, threads it into the loader and the search backend, and points
`config` at it for the duration of each call so the parts that reach for
`config` directly - the `TypeResolver` singleton, which has to find `types/` -
follow too. A test proves no path of the developer's checkout is read while a
foreign root is set.
**Values and exceptions, not exit codes.** The functions return the same
structures the CLI's `--json` forms print - one wire contract, with the CLI as
its executable specification - and raise `ChemenuError` where the CLI would
print `ERROR` and leave through `typer.Exit(1)`.
Read-only, structurally: nothing under `chemenu.commands` is imported, so `new`,
`touch`, `xref`, `cite`, `publish`, `migrate` and `version bump` are not
reachable from here at all. That is the property #19 asks for - the write
functions do not exist in this surface rather than being filtered out of it.
Import cost is the whole read core and nothing else: `yaml` and `jsonschema`,
plus the standard library. No `typer`, no `rich`.
"""
from __future__ import annotations
import datetime
from pathlib import Path
from typing import Any, Iterable, Optional
from chemenu import config
from chemenu.corpus_cache import CorpusCache
from chemenu.errors import BackendError, ChemenuError, ValidationError
from chemenu.lint_core import run_lint
from chemenu.search import filters
from chemenu.search.registry import resolve
from chemenu.search.service import run_search, unreadable_pages
from chemenu.search.types import Predicate, SearchQuery
from chemenu.types_core import describe_type, list_types
# Distinguishes "the caller did not pass a revision" from "the caller passed
# None", which is itself a meaningful answer: no commit, because the tree is
# dirty or is not a checkout.
_UNREAD = object()
__all__ = [
"Corpus",
"ChemenuError",
"ValidationError",
"BackendError",
]
class Corpus:
"""One corpus tree, read repeatedly.
`root` follows `config.resolve_root()`: an explicit path, else
`$CHEMENU_ROOT`, else the checkout this package lives in. `kb_dir` defaults
to `<root>/kb` and is separate only because the search backend already
distinguishes the two.
Holds a `CorpusCache`, so a long-lived caller parses the corpus once per
commit instead of once per request - and never answers from a superseded
parse, because a dirty tree is not cached. Not thread-safe: a server serving
concurrent requests holds the lock, for the reason given in
`chemenu/corpus_cache.py`.
"""
def __init__(self, root: Optional[Path | str] = None, kb_dir: Optional[Path | str] = None):
self.root = config.resolve_root(root)
self.kb_dir = Path(kb_dir) if kb_dir is not None else self.root / "kb"
self._cache = CorpusCache(self.kb_dir, self.root)
@property
def revision(self) -> Optional[str]:
"""The commit every answer from this corpus is stamped with, or None
when the tree is dirty or is not a git checkout - in which case the
answer corresponds to no commit, and says so."""
return self._cache.current_revision()
def _rooted(self):
"""Point `config` at this corpus for the duration of one call.
Threading a root through every argument gets the search backend and the
corpus loader, and misses the module-level `TypeResolver` singleton -
which resolves `types/` and is what `Page.kind` goes through. Without
this, a foreign corpus is read with *this* checkout's type specs, and
the page's `kind` is an answer about the wrong instance.
Process-wide while open, so `Corpus` inherits `config.rooted()`'s
thread-safety constraint: one lock per process, held by the caller.
"""
return config.rooted(self.root)
def _stamp(
self, payload: dict[str, Any], revision: Optional[str] = _UNREAD
) -> dict[str, Any]:
"""Every response carries the revision it was computed from.
A stale checkout otherwise answers confidently and wrongly, which is the
failure `SOUL.md` names as the cardinal one. The stamp turns a silent
stale answer into a visible one.
A caller that loaded the corpus passes the revision it got back, rather
than letting this ask again: between the load and the stamp the tree can
move, and the honest answer is the revision the pages actually came
from. Callers that read no pages (`types`) ask for the current one.
"""
if revision is _UNREAD:
revision = self._cache.current_revision()
payload["commit"] = revision
payload["as_of"] = datetime.datetime.now(datetime.timezone.utc).isoformat()
return payload
def search(
self,
text: Optional[str] = None,
predicates: Iterable[str] = (),
regex: bool = False,
limit: int = 20,
sort: Optional[str] = None,
backend: Optional[str] = None,
) -> dict[str, Any]:
"""`wikitool search --json`, as a value.
`predicates` takes the raw `--field` strings, so the CLI and this share
one parser and cannot drift on what `confidence<0.6` means.
"""
raw = list(predicates)
if not text and not raw:
raise ValidationError(
"Nothing to search for: give a query, or at least one predicate."
)
parsed: tuple[Predicate, ...] = tuple(filters.parse_predicate(p) for p in raw)
backends = resolve(backend, self.kb_dir, self.root)
query = SearchQuery(
text=text, predicates=parsed, regex=regex, limit=limit, sort=sort
)
with self._rooted():
pages, revision = self._cache.load()
hits = run_search(query, pages, backends, self.kb_dir)
return self._stamp({
"query": text,
"predicates": [p.render() for p in parsed],
"backend": ",".join(b.name for b in backends),
"count": len(hits),
"results": [hit.as_dict() for hit in hits],
"unreadable": unreadable_pages(pages),
}, revision)
def lint(self) -> dict[str, Any]:
"""`wikitool lint --json`, as a value.
The JSON form only - `lint` without a flag writes a report into
`reports/`, and a read surface does not write into the tree it is
reading.
"""
with self._rooted():
return self._stamp(run_lint(self.kb_dir))
def types(self) -> dict[str, Any]:
"""`wikitool types list --json`, as a value."""
with self._rooted():
return self._stamp({"types": list_types()})
def describe_type(self, name: str) -> dict[str, Any]:
"""`wikitool types describe <name> --json`, as a value. Raises
`UnknownType` (a `ValidationError`) for a name that does not exist."""
with self._rooted():
return self._stamp(describe_type(name))
def status(self) -> dict[str, Any]:
"""A composed snapshot: how big the corpus is and what lint says about
it, without the full report.
Composed here on purpose. There is no `wikitool status` to wrap -
`wiki-status` is a *skill* that assembles `kb/index.md`, `lint` and
`kb/log.md` - so this is a new surface, and saying so is what keeps
anyone from looking for the CLI command it mirrors.
"""
with self._rooted():
pages, revision = self._cache.load()
report = run_lint(self.kb_dir)
collections: dict[str, int] = {}
for page in pages.values():
name = filters.collection_of(page.path, self.kb_dir)
if name:
collections[name] = collections.get(name, 0) + 1
return self._stamp({
"pages": len(pages),
"collections": dict(sorted(collections.items())),
"findings": {
key: len(value)
for key, value in sorted(report.items())
if isinstance(value, list)
},
"unreadable": unreadable_pages(pages),
}, revision)
+136
View File
@@ -0,0 +1,136 @@
"""Generated regions inside a page body, found by delimiter rather than by prose.
`xref` owns the links block and `cite` owns the footnotes block. Both used to be
located by matching their **heading text** - `^## Beziehungen$` - which made a
translated heading a structural fact and put the KB language into the compiler.
It also made the region's *end* a guess: the footnotes block ran to the next
heading, and before that to the end of the file, which silently deleted whatever
sat after it on eight pages.
A marker pair answers both questions exactly:
<!-- wikitool:links -->
## Beziehungen
- **depends-on:** [[Hermes]]
<!-- /wikitool:links -->
Everything between the markers is generated and is replaced wholesale on the
next write - heading included, which is why the heading text is a *rendering*
value from `kb/CONVENTIONS.md` rather than something the tool searches for. An
author never edits inside the markers; anything they put there is overwritten
without warning, exactly like `kb/index.md`.
The markers are HTML comments: invisible in every renderer this corpus is read
through, and the same convention `dist:strip-start`/`-end` already uses in
`AGENTS.md`. They cost a reader nothing and cost an LLM about twenty tokens a
page - the price of not having to guess where a generated region ends.
"""
from __future__ import annotations
import re
from typing import Optional
# The two regions the tool owns. `see-also` is deliberately absent: it was the
# reciprocal half of the old bidirectional `xref add`, and under authored
# directional edges it is a *label* (`see-also`) inside the links block, not a
# section of its own.
LINKS = "links"
FOOTNOTES = "footnotes"
BLOCKS = (LINKS, FOOTNOTES)
_NAME = r"[a-z][a-z0-9-]*"
def open_marker(name: str) -> str:
return f"<!-- wikitool:{name} -->"
def close_marker(name: str) -> str:
return f"<!-- /wikitool:{name} -->"
def _region_re(name: str) -> re.Pattern[str]:
"""The whole region including both markers and the blank line around it."""
return re.compile(
r"\n*"
+ re.escape(open_marker(name))
+ r".*?"
+ re.escape(close_marker(name))
+ r"\n*",
re.DOTALL,
)
_ANY_OPEN_RE = re.compile(rf"<!-- wikitool:({_NAME}) -->")
_ANY_CLOSE_RE = re.compile(rf"<!-- /wikitool:({_NAME}) -->")
def find(body: str, name: str) -> Optional[str]:
"""The generated content of `name`'s region, markers excluded, or None."""
match = _region_re(name).search(body)
if not match:
return None
text = match.group(0)
start = text.index(open_marker(name)) + len(open_marker(name))
end = text.index(close_marker(name))
return text[start:end].strip("\n")
def render(name: str, heading: str, lines: list[str]) -> str:
"""A whole region, ready to place into a body. Empty `lines` renders "".
An empty region is no region at all rather than a heading with nothing under
it: a page that cites nothing should not carry an empty Footnotes section,
and the same holds for a page with no declared edges.
"""
if not lines:
return ""
parts = [open_marker(name), f"## {heading}", "", *lines, close_marker(name)]
return "\n".join(parts)
def replace(body: str, name: str, region: str) -> str:
"""Put `region` where `name`'s region is, or append it if there is none.
Appending at the end is right for both blocks: they are the page's trailing
machine-owned material, and an author's prose never follows them. A region
that is `""` removes what was there.
"""
existing = _region_re(name).search(body)
if existing:
replacement = f"\n\n{region}\n" if region else "\n"
return (body[: existing.start()] + replacement + body[existing.end():]).rstrip("\n") + "\n"
if not region:
return body
return body.rstrip("\n") + "\n\n" + region + "\n"
def strip(body: str, name: str) -> str:
"""The body with `name`'s region removed entirely."""
return replace(body, name, "")
def marker_pairs(body: str) -> dict[str, int]:
"""How many complete open/close pairs each region name has in `body`.
The invariant `migrate verify` checks. An agent rewriting prose next to a
boundary can drop or duplicate a marker, and the failure is otherwise silent:
a lost opening marker turns a generated region into ordinary prose that the
next write appends a second copy beside.
"""
opens = [m.group(1) for m in _ANY_OPEN_RE.finditer(body)]
closes = [m.group(1) for m in _ANY_CLOSE_RE.finditer(body)]
names = set(opens) | set(closes)
return {name: min(opens.count(name), closes.count(name)) for name in sorted(names)}
def unbalanced_markers(body: str) -> list[str]:
"""Region names whose open and close markers do not pair up."""
opens = [m.group(1) for m in _ANY_OPEN_RE.finditer(body)]
closes = [m.group(1) for m in _ANY_CLOSE_RE.finditer(body)]
return sorted(
name
for name in set(opens) | set(closes)
if opens.count(name) != closes.count(name)
)
+2
View File
@@ -21,6 +21,7 @@ try:
index_build, index_build,
instructions_cmd, instructions_cmd,
lint as lint_module, lint as lint_module,
links_cmd,
log_append, log_append,
migrate_cmd, migrate_cmd,
new_page, new_page,
@@ -54,6 +55,7 @@ app = typer.Typer(
app.add_typer(xref.app, name="xref") app.add_typer(xref.app, name="xref")
app.add_typer(cite_cmd.app, name="cite") app.add_typer(cite_cmd.app, name="cite")
app.add_typer(links_cmd.app, name="links")
app.add_typer(index_build.app, name="index") app.add_typer(index_build.app, name="index")
app.add_typer(log_append.app, name="log") app.add_typer(log_append.app, name="log")
app.add_typer(confidence_decay.app, name="confidence") app.add_typer(confidence_decay.app, name="confidence")
+2 -3
View File
@@ -28,7 +28,6 @@ from chemenu.kb_scan import load_kb_pages
from chemenu.provenance import ( from chemenu.provenance import (
CITE_REF_RE, CITE_REF_RE,
cite_id, cite_id,
cite_block_heading,
render_page_body, render_page_body,
split_cite_block, split_cite_block,
unique_cite_id, unique_cite_id,
@@ -81,7 +80,7 @@ def upsert_citation(page: Page, source_title: str, qualifier: Optional[str]) ->
if sources_changed: if sources_changed:
sources.append(source_title) sources.append(source_title)
new_body = render_page_body(head, definitions, cite_block_heading(page.body)) new_body = render_page_body(head, definitions)
changed = block_changed or sources_changed or new_body != page.body changed = block_changed or sources_changed or new_body != page.body
return marker_id, new_body, changed return marker_id, new_body, changed
@@ -152,7 +151,7 @@ def sync_page(page: Page) -> tuple[str, bool, list[str], list[str]]:
ordered[cid] = definitions[cid] ordered[cid] = definitions[cid]
seen.add(cid) seen.add(cid)
new_body = render_page_body(head, ordered, cite_block_heading(page.body)) new_body = render_page_body(head, ordered)
changed = new_body != page.body changed = new_body != page.body
return new_body, changed, pruned, undefined return new_body, changed, pruned, undefined
+117 -10
View File
@@ -2,9 +2,13 @@
repo's machinery. repo's machinery.
`export` copies the pipeline's schema/compiler/control-plane layers (types/, `export` copies the pipeline's schema/compiler/control-plane layers (types/,
tools/, instructions/, the stage contracts, every kb/*/COLLECTION.md) into an tools/, instructions/, the stage contracts) into an empty target, with no kb/
empty target, with no kb/ pages, no raw/ content, and no git history - see pages, no raw/ content, and no git history - see instructions/setup-instance.md
instructions/setup-instance.md for what happens after. It never calls git. for what happens after. It never calls git.
The two binding-but-instance-owned documents under kb/ - each collection's
COLLECTION.md and kb/CONVENTIONS.md - cross as `.template` and are adopted by a
rename, the same split USER.md/SOUL.md use at the repo root.
Three independent exclusion mechanisms feed the plan, for three different Three independent exclusion mechanisms feed the plan, for three different
shapes of "does not belong in someone else's instance": shapes of "does not belong in someone else's instance":
@@ -35,7 +39,7 @@ from typing import Callable, NamedTuple, Optional, Union
import typer import typer
from chemenu import config, kb_collections, kb_state, version as version_mod from chemenu import config, conventions, kb_collections, kb_state, version as version_mod
from chemenu.commands._util import fail, rel_path, success, today_iso from chemenu.commands._util import fail, rel_path, success, today_iso
app = typer.Typer(help="Build a distributable copy of the wiki machinery.") app = typer.Typer(help="Build a distributable copy of the wiki machinery.")
@@ -64,8 +68,14 @@ DIST_TEMPLATES_DIR = Path(__file__).resolve().parent.parent / "dist_templates"
# and does not load `AGENTS.md`, so it ships for the same reason # and does not load `AGENTS.md`, so it ships for the same reason
# `.claude/settings.json` does - a distributed instance running that harness # `.claude/settings.json` does - a distributed instance running that harness
# would otherwise start every session without the control plane. # would otherwise start every session without the control plane.
#
# `INSTALL-MCP.md` ships beside `INSTALL.md` and for the same reason: the MCP
# read server is part of what an instance *has*, even though its dependency is
# optional. A distribution whose server is present but undocumented is one
# whose operator finds the module by reading the source.
ROOT_FILES = ( ROOT_FILES = (
"AGENTS.md", "CLAUDE.md", "README.md", "EVALS.md", "INSTALL.md", ".gitignore", "VERSION", "AGENTS.md", "CLAUDE.md", "README.md", "EVALS.md", "INSTALL.md", "INSTALL-MCP.md",
".gitignore", "VERSION",
*config.LICENSE_FILES, *config.LICENSE_FILES,
*config.PERSONALIZATION_TEMPLATES, *config.PERSONALIZATION_TEMPLATES,
config.ENVIRONMENT_TEMPLATE, config.ENVIRONMENT_TEMPLATE,
@@ -254,6 +264,60 @@ class Origin(NamedTuple):
update_url: Optional[str] = None update_url: Optional[str] = None
def instance_owned_type_stems() -> set[str]:
"""Type-spec stems whose instances are knowledge pages, and which therefore
belong to the instance rather than to the stack.
The line is `root:`, and it was already in the frontmatter before anyone
drew it: `root: kb` means the type describes a page the instance writes, so
its prose, its template and its language are the instance's business.
Anything else - `instruction` (`root: repo`), `lint-report` (no `base_dir`
at all), `type-spec` itself - describes a stack artifact and ships verbatim.
Read from `types/` rather than listed, so an instance adding its own page
type gets the same treatment without a code change.
"""
from chemenu.type_resolver import resolver
stems: set[str] = set()
for type_path, frontmatter in resolver.list_type_specs():
stem = Path(type_path).stem
if stem == "type-spec":
continue
if not frontmatter.get("base_dir"):
continue
if (frontmatter.get("root") or "kb") != "kb":
continue
stems.add(stem)
return stems
def _plan_types() -> dict[str, PlannedFile]:
"""`types/`, with the page type-specs re-keyed as templates.
Same split as the collection contracts, for the same reason and by the same
mechanism: the shipped content is a working default rather than something
wrong for the receiver, so the file itself crosses - under a name that has
to be adopted before it counts. A type-spec's `.schema.yaml` travels with
it, because the two are one type (see types/type-spec.md § Anatomy) and
adopting half of it would leave a spec validated by a file it does not own.
"""
plan = _copy_tree(config.TYPES_DIR, "types", frozenset())
stems = instance_owned_type_stems()
if not stems:
return plan
rekeyed: dict[str, PlannedFile] = {}
for relative, planned in plan.items():
name = relative.rsplit("/", 1)[-1]
stem = name.split(".", 1)[0]
if stem in stems:
rekeyed[f"{relative}.template"] = planned
else:
rekeyed[relative] = planned
return rekeyed
def build_plan(origin: Optional[Origin] = None) -> dict[str, PlannedFile]: def build_plan(origin: Optional[Origin] = None) -> dict[str, PlannedFile]:
"""Every (destination-relative path -> planned file) the export writes.""" """Every (destination-relative path -> planned file) the export writes."""
plan: dict[str, PlannedFile] = {} plan: dict[str, PlannedFile] = {}
@@ -276,19 +340,39 @@ def build_plan(origin: Optional[Origin] = None) -> dict[str, PlannedFile]:
plan[name] = _read_planned_file(source, name) plan[name] = _read_planned_file(source, name)
plan.update(_copy_tree(config.INSTRUCTIONS_DIR, "instructions", frozenset(INSTRUCTIONS_EXCLUDE_DIRS))) plan.update(_copy_tree(config.INSTRUCTIONS_DIR, "instructions", frozenset(INSTRUCTIONS_EXCLUDE_DIRS)))
plan.update(_copy_tree(config.TYPES_DIR, "types", frozenset())) plan.update(_plan_types())
plan.update(_copy_tree( plan.update(_copy_tree(
config.ROOT / "tools", "tools", frozenset(TOOLS_EXCLUDE_DIRS), _is_coverage_output config.ROOT / "tools", "tools", frozenset(TOOLS_EXCLUDE_DIRS), _is_coverage_output
)) ))
for hook_dir in HOOK_DIRS: for hook_dir in HOOK_DIRS:
plan.update(_copy_tree(config.ROOT / hook_dir, hook_dir, frozenset())) plan.update(_copy_tree(config.ROOT / hook_dir, hook_dir, frozenset()))
# `kb/CONTRACT.md` is stack-owned and ships verbatim; everything beside it
# under `kb/` is the instance's own and ships only as a `.template`. That is
# the personalization split (`USER.md`/`SOUL.md`) one directory down, and
# the reason for it is the same: a distribution can say what the file
# decides, never what this instance decided.
kb_contract = config.KB_DIR / "CONTRACT.md" kb_contract = config.KB_DIR / "CONTRACT.md"
if kb_contract.is_file(): if kb_contract.is_file():
plan["kb/CONTRACT.md"] = _read_planned_file(kb_contract, "kb/CONTRACT.md") plan["kb/CONTRACT.md"] = _read_planned_file(kb_contract, "kb/CONTRACT.md")
conventions_template = config.KB_DIR / conventions.CONVENTIONS_TEMPLATE
if conventions_template.is_file():
rel = f"kb/{conventions.CONVENTIONS_TEMPLATE}"
plan[rel] = _read_planned_file(conventions_template, rel)
# A collection contract is instance-owned too, but unlike `USER.md` the
# shipped content is not *wrong* for the receiver - it is the profile this
# repo's own collections adopted, and a fine starting point. So the file
# itself ships, under the template name: one source of truth here, and a
# receiving instance that has to rename it before it counts. Keeping a
# separate `.template` beside each contract would have meant maintaining two
# near-identical copies of the same text, which is the drift AGENTS.md
# invariant 8 exists to prevent.
for collection in kb_collections.iter_kb_collections(): for collection in kb_collections.iter_kb_collections():
rel = f"kb/{collection.name}/COLLECTION.md" source = collection / kb_collections.CONTRACT_NAME
plan[rel] = _read_planned_file(collection / "COLLECTION.md", rel) rel = f"kb/{collection.name}/{kb_collections.CONTRACT_NAME}.template"
plan[rel] = _read_planned_file(source, rel)
for relative in CONTRACT_ONLY_STAGES: for relative in CONTRACT_ONLY_STAGES:
source = config.ROOT / relative source = config.ROOT / relative
@@ -333,17 +417,39 @@ def build_plan(origin: Optional[Origin] = None) -> dict[str, PlannedFile]:
# IP literals) was considered and rejected - the project's own host legitimately # IP literals) was considered and rejected - the project's own host legitimately
# appears in INSTALL.md and version.py, so such a scan would either whitelist # appears in INSTALL.md and version.py, so such a scan would either whitelist
# the very string it is looking for or cry wolf on every export. # the very string it is looking for or cry wolf on every export.
#
# `COLLECTION.md` and `CONVENTIONS.md` are deliberately *not* on the allowed
# list any more. Both bind, and both are the instance's to write, so they cross
# the boundary as `.template` and are adopted by a rename - a plan carrying the
# filled name would hand a new instance this one's authoring conventions as
# though they were the stack's.
_CONTENT_PREFIXES = ("kb/", "raw/") _CONTENT_PREFIXES = ("kb/", "raw/")
_CONTENT_ALLOWED_NAMES = ("CONTRACT.md", "COLLECTION.md", "log.md", ".gitkeep") _CONTENT_ALLOWED_NAMES = (
"CONTRACT.md",
f"{kb_collections.CONTRACT_NAME}.template",
conventions.CONVENTIONS_TEMPLATE,
"log.md",
".gitkeep",
)
_INSTANCE_OWNED_KB_FILES = (kb_collections.CONTRACT_NAME, conventions.CONVENTIONS_FILENAME)
def find_leaks(plan: dict[str, PlannedFile]) -> list[str]: def find_leaks(plan: dict[str, PlannedFile]) -> list[str]:
"""Planned paths that carry one instance's own data instead of machinery.""" """Planned paths that carry one instance's own data instead of machinery."""
owned_types = instance_owned_type_stems()
leaks: list[str] = [] leaks: list[str] = []
for relative in sorted(plan): for relative in sorted(plan):
name = relative.rsplit("/", 1)[-1] name = relative.rsplit("/", 1)[-1]
if name in config.PERSONALIZATION_FILES or name == config.ENVIRONMENT_FILE: if name in config.PERSONALIZATION_FILES or name == config.ENVIRONMENT_FILE:
leaks.append(f"{relative} (one instance's own personalization)") leaks.append(f"{relative} (one instance's own personalization)")
elif relative.startswith("kb/") and name in _INSTANCE_OWNED_KB_FILES:
leaks.append(f"{relative} (this instance's authoring conventions; ship the .template)")
elif (
relative.startswith("types/")
and not relative.endswith(".template")
and name.split(".", 1)[0] in owned_types
):
leaks.append(f"{relative} (this instance's page type-spec; ship the .template)")
elif relative.startswith("instructions/dev/"): elif relative.startswith("instructions/dev/"):
leaks.append(f"{relative} (stack-development only)") leaks.append(f"{relative} (stack-development only)")
elif relative.startswith(_CONTENT_PREFIXES) and name not in _CONTENT_ALLOWED_NAMES: elif relative.startswith(_CONTENT_PREFIXES) and name not in _CONTENT_ALLOWED_NAMES:
@@ -388,7 +494,8 @@ def export_command(
AGENTS.md/README.md (dev-instance-only marker blocks removed), AGENTS.md/README.md (dev-instance-only marker blocks removed),
instructions/ (no instructions/dev/), types/, tools/ (no venv/caches), instructions/ (no instructions/dev/), types/, tools/ (no venv/caches),
the .github/hooks/+.vibe session-tracing config plus .claude/settings.json, the .github/hooks/+.vibe session-tracing config plus .claude/settings.json,
every kb/*/COLLECTION.md (no pages, no areas), empty kb/CONTRACT.md plus a COLLECTION.md.template per collection and
kb/CONVENTIONS.md.template (no pages, no areas), empty
raw/{articles,documents,notes,assets}/, VERSION, the USER.md/SOUL.md raw/{articles,documents,notes,assets}/, VERSION, the USER.md/SOUL.md
personalization templates (never the filled files), and a personalization templates (never the filled files), and a
.wikitool-release.json stamp. The --source-*/--release-url/--update-url .wikitool-release.json stamp. The --source-*/--release-url/--update-url
+101 -2
View File
@@ -37,7 +37,7 @@ from typing import Optional
import typer import typer
from chemenu import config, kb_collections, version as version_mod from chemenu import config, conventions, kb_collections, version as version_mod
from chemenu.commands._util import fail, rel_path, success from chemenu.commands._util import fail, rel_path, success
app = typer.Typer(help="Verify documentation that mirrors the code or repo layout.") app = typer.Typer(help="Verify documentation that mirrors the code or repo layout.")
@@ -114,6 +114,12 @@ REQUIRED_TRACKED_PATHS = (
"instructions/CONTRACT.md", "instructions/CONTRACT.md",
"instructions/wiki-query/SKILL.md", "instructions/wiki-query/SKILL.md",
"ENVIRONMENT.md.template", "ENVIRONMENT.md.template",
# The one `.template` that lives under a content directory. It is what a
# distribution ships in place of this instance's own `kb/CONVENTIONS.md`, so
# an ignore rule reaching it would produce exports whose receiving instance
# has nothing to fill in - and `find_leaks` refuses to substitute the filled
# file, correctly, so the export would simply be missing it.
"kb/CONVENTIONS.md.template",
) )
CLI_README = config.ROOT / "tools" / "CONTRACT.md" CLI_README = config.ROOT / "tools" / "CONTRACT.md"
@@ -207,13 +213,22 @@ def check_cli_readme() -> list[str]:
def check_collection_contracts() -> list[str]: def check_collection_contracts() -> list[str]:
"""The three structural rules that define what a collection is. """The structural rules that define what a collection is, plus what each one
has to declare about itself.
Collections are discovered by contract presence rather than listed here, so Collections are discovered by contract presence rather than listed here, so
`mkdir kb/<name>` + a COLLECTION.md is all it takes to add one. That only `mkdir kb/<name>` + a COLLECTION.md is all it takes to add one. That only
works if the inverse is also checked: a directory under kb/ *without* a works if the inverse is also checked: a directory under kb/ *without* a
contract is an unclaimed subtree whose pages obey no local rules, and a contract is an unclaimed subtree whose pages obey no local rules, and a
contract outside kb/ quietly widens "collection" back out to "any directory". contract outside kb/ quietly widens "collection" back out to "any directory".
Presence alone stopped being enough once the contracts became
instance-owned. A `COLLECTION.md` an instance wrote can be about anything,
so the two facts the stack still needs from it - which profile it adopted,
and whether the stack resolves against it by name - are declared in its
frontmatter and checked here (`kb_collections.declaration_issues`), together
with the shape of `kb/CONVENTIONS.md`, whose section names the compiler
reads.
""" """
issues = [] issues = []
@@ -243,6 +258,52 @@ def check_collection_contracts() -> list[str]:
if not (config.ROOT / relative_path).exists(): if not (config.ROOT / relative_path).exists():
issues.append(f"{relative_path} is missing - it is the authoring contract for its stage") issues.append(f"{relative_path} is missing - it is the authoring contract for its stage")
issues += kb_collections.declaration_issues()
issues += conventions.declaration_issues()
issues += check_stack_required_types()
return issues
def check_stack_required_types() -> list[str]:
"""The minimum the stack asks of the type layer, and nothing beyond it.
The four page type-specs belong to the instance: it may translate them,
rewrite their templates, add sections. What it may not do is remove the one
type the provenance path is built on, or drop the field that path reads.
Everything else about `types/source.md` - its prose, its template, its title
prefix, its directory - is the instance's, and is deliberately not checked
here.
"""
from chemenu.type_resolver import resolver
issues: list[str] = []
for type_name in kb_collections.STACK_REQUIRED_TYPES:
try:
type_path = resolver.find_type_by_name(type_name)
except (ValueError, OSError) as exc:
issues.append(f"types/ could not be read to find the `{type_name}` type: {exc}")
continue
if not type_path:
issues.append(
f"no type-spec declares `name: {type_name}` - `sources coverage`, `[^cite-id]` "
f"resolution and `kb/provenance.md` all ask `page.kind == \"{type_name}\"`, so "
f"without it the whole raw/ -> kb/ provenance path resolves against nothing"
)
continue
try:
schema = resolver.get_schema(type_path) or {}
except (ValueError, OSError) as exc:
issues.append(f"{type_path}: its schema could not be read: {exc}")
continue
declared = set(schema.get("required") or [])
for field in kb_collections.STACK_REQUIRED_TYPE_FIELDS.get(type_name, ()):
if field not in declared:
issues.append(
f"{type_path}: its schema must require `{field}` - it is what the "
f"provenance path reads, and a `{type_name}` page without it claims no "
f"raw material at all"
)
return issues return issues
@@ -473,6 +534,43 @@ def check_migration_for_boundary() -> list[str]:
] ]
def check_breaking_change_for_boundary() -> list[str]:
"""A version that crosses the compatibility boundary must say what breaks.
Separate from `check_migration_for_boundary`, because the two answer
different questions: that one asks whether the *content* has to move, this
one whether the operator was told the swap is not drop-in at all. A
boundary crossing with an untouched corpus - a renamed feed, artefact,
import name or flag - satisfies that check and still leaves every existing
instance with something to do by hand.
Only the newest entry is checked, for the same reason: older crossings are
history, and re-reporting them forever would make the check noise.
"""
changes_path = config.ROOT / version_mod.CHANGES_FILENAME
version_path = config.ROOT / version_mod.VERSION_FILENAME
if not changes_path.is_file() or not version_path.is_file():
return [] # already reported by check_version_changelog
text = changes_path.read_text(encoding="utf-8")
current = version_mod.top_changes_version(text)
previous = _second_changes_version(text)
if current is None or previous is None:
return [] # the first versioned entry has no predecessor to cross from
if current.compat_key == previous.compat_key:
return []
if version_mod.BREAKING_CHANGE_MARKER in (version_mod.changes_section(text, current) or ""):
return []
return [
f"{current} crosses the compatibility boundary from {previous}, so it is not a drop-in "
f"replacement - but its {version_mod.CHANGES_FILENAME} entry carries no "
f"`{version_mod.BREAKING_CHANGE_MARKER}` line saying what stops working. Add it "
"(`version bump --breaking` writes it; see instructions/dev/version-parts.md)"
]
@app.command("verify") @app.command("verify")
def verify(): def verify():
"""Check the CLI/README command tables, contract presence, type-form drift, ignore rules, and version/changelog agreement.""" """Check the CLI/README command tables, contract presence, type-form drift, ignore rules, and version/changelog agreement."""
@@ -484,6 +582,7 @@ def verify():
+ check_ignored_content() + check_ignored_content()
+ check_version_changelog() + check_version_changelog()
+ check_migration_for_boundary() + check_migration_for_boundary()
+ check_breaking_change_for_boundary()
) )
if issues: if issues:
+96 -5
View File
@@ -20,8 +20,8 @@ from typing import Optional
import typer import typer
from rich.console import Console from rich.console import Console
from chemenu import config, kb_collections, version as version_mod from chemenu import config, conventions, kb_collections, version as version_mod
from chemenu.commands import instructions_cmd from chemenu.commands import git_publish, instructions_cmd
from chemenu.commands._util import rel_path from chemenu.commands._util import rel_path
from chemenu.session import ENV_VAR as SESSION_ENV_VAR from chemenu.session import ENV_VAR as SESSION_ENV_VAR
@@ -213,6 +213,50 @@ def check_personalization() -> Check:
return Check("personalization", "OK", f"{', '.join(config.PERSONALIZATION_FILES)} present and filled") return Check("personalization", "OK", f"{', '.join(config.PERSONALIZATION_FILES)} present and filled")
def check_conventions() -> Check:
"""Whether this instance has said how its own pages are written.
`kb/CONVENTIONS.md` carries the decisions `kb/CONTRACT.md` deliberately no
longer makes: the KB language and the headings its two generated regions
render under, the tone examples, the confidence rubric, the naming forms.
`FAIL` rather than `WARN` because those decisions bind every page, and
because it has the same two failure modes the personalization pair has: the
distribution can ship the template but never the filled file, so a template
renamed and left unanswered looks present and decides nothing.
The headings themselves are only cosmetic now - the marker pair carries each
region's identity, so a default renders wrong words rather than corrupting
structure. That is why this check is about the *file*, not about rescuing a
lookup the compiler can no longer get wrong.
"""
path = conventions.conventions_file()
fix = (
"Copy kb/CONVENTIONS.md.template to kb/CONVENTIONS.md and answer it - the KB-language "
"step of instructions/setup-instance.md walks it, and instructions/kb-profiles.md has "
"the ready-made profiles to adopt"
)
if not path.is_file():
return Check(
"conventions", "FAIL",
f"kb/{conventions.CONVENTIONS_FILENAME} is missing - this instance has not "
"declared how its pages are written",
fix,
)
issues = conventions.declaration_issues()
if issues:
return Check("conventions", "FAIL", "; ".join(issues), fix)
declared = conventions.language() or "unspecified"
from chemenu import blocks
headings = ", ".join(conventions.heading(block) for block in blocks.BLOCKS)
return Check(
"conventions", "OK",
f"kb/{conventions.CONVENTIONS_FILENAME} present, language {declared}, "
f"sections {headings}",
)
def check_environment() -> Check: def check_environment() -> Check:
"""Whether this checkout records the environment it works through. """Whether this checkout records the environment it works through.
@@ -242,6 +286,51 @@ def check_environment() -> Check:
return Check("environment", "OK", f"{config.ENVIRONMENT_FILE} present and filled") return Check("environment", "OK", f"{config.ENVIRONMENT_FILE} present and filled")
def check_publish_remotes() -> Check:
"""Whether the Publish-Remote Gate is armed in this checkout.
Absent is a legitimate state, not a fault: a checkout with a single remote
and nothing private in it has nothing to protect, and making the file
mandatory would turn a safeguard into paperwork. So this never FAILs - it
reports, the way `environment` does.
It does WARN for the case that actually bites: more than one remote
configured and no allowlist. That is the shape a private instance has after
it adds the public upstream, and it is exactly when a wrong `--remote`
stops being a typo and starts being a disclosure.
Both absent states say **armed** or **not armed** rather than only naming
the file. AGENTS.md lists this among the three limits enforced in code, so a
line that reports the file's absence and leaves the reader to infer what
that means about the gate is how a checkout ends up trusting a safeguard
that is not running - which is worse than having none.
"""
urls = git_publish.read_allowed_push_urls()
if urls is not None:
return Check(
"publish-remotes", "OK",
f"Gate armed: {len(urls)} allowed push target(s) in "
f"{config.PUBLISH_REMOTES_FILENAME}",
)
result = subprocess.run(
["git", "remote"], cwd=config.ROOT, capture_output=True, text=True
)
remotes = [r for r in result.stdout.split() if r]
if len(remotes) > 1:
return Check(
"publish-remotes", "WARN",
f"Gate not armed: {len(remotes)} remotes ({', '.join(remotes)}) and no "
f"{config.PUBLISH_REMOTES_FILENAME} - every one of them is a legal publish target",
f"Create {config.PUBLISH_REMOTES_FILENAME} naming the push URL this checkout "
"may publish to - see instructions/gates.md",
)
return Check(
"publish-remotes", "OK",
f"Gate not armed: no {config.PUBLISH_REMOTES_FILENAME} - any push target passes "
"(1 remote configured, nothing to confuse it with)",
)
def check_generated_files() -> Check: def check_generated_files() -> Check:
missing = [ missing = [
rel_path(path) rel_path(path)
@@ -354,7 +443,9 @@ def run_doctor() -> list[Check]:
check_skills(), check_skills(),
check_structure(), check_structure(),
check_personalization(), check_personalization(),
check_conventions(),
check_environment(), check_environment(),
check_publish_remotes(),
check_generated_files(), check_generated_files(),
check_session_id(), check_session_id(),
] ]
@@ -365,9 +456,9 @@ def doctor_command(
json_out: bool = typer.Option(False, "--json", help="Print the checks as JSON"), json_out: bool = typer.Option(False, "--json", help="Print the checks as JSON"),
): ):
"""Check that this instance is correctly configured: dependencies, author, """Check that this instance is correctly configured: dependencies, author,
git identity/remote, published skills, structure, personalization, git identity/remote, published skills, structure, personalization, KB
generated files, and session scoping. Read-only. Exits 1 only if a check conventions, generated files, and session scoping. Read-only. Exits 1 only
FAILs.""" if a check FAILs."""
checks = run_doctor() checks = run_doctor()
if json_out: if json_out:
+100
View File
@@ -80,6 +80,86 @@ def _run(args: list[str]) -> subprocess.CompletedProcess:
return subprocess.run(args, cwd=config.ROOT, capture_output=True, text=True) return subprocess.run(args, cwd=config.ROOT, capture_output=True, text=True)
# --- Publish-Remote Gate -----------------------------------------------------
#
# The Mass-Update Gate asks "is this too much to publish?". This one asks the
# question underneath it: "is this the right place to publish to at all?".
#
# A checkout holding private content typically has two remotes - its own, and
# the public upstream it takes stack updates from. Nothing in git distinguishes
# them at push time, so a single wrong `--remote` puts a private corpus on a
# public repository, where a force-push does not take it back: the objects stay
# fetchable by SHA until someone expires the server's reflogs.
#
# Like the other two gates this refuses with exit 42 and has **no flag that
# opens it**. The way past it is to name the URL in the file, which is an edit
# the user makes deliberately rather than something an agent can decide mid-run.
def read_allowed_push_urls() -> Optional[list[str]]:
"""The URLs this checkout permits `publish` to push to, or None when the
file is absent (unrestricted - see config.PUBLISH_REMOTES_FILENAME)."""
path = config.ROOT / config.PUBLISH_REMOTES_FILENAME
if not path.is_file():
return None
try:
data = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
fail(
f"{config.PUBLISH_REMOTES_FILENAME} is unreadable ({exc}). It decides where "
"`publish` may push, so a broken one is not treated as 'no restriction' - "
"fix the file or delete it deliberately."
)
return None
urls = data.get("allowed_push_urls")
if not isinstance(urls, list) or not all(isinstance(u, str) for u in urls):
fail(
f"{config.PUBLISH_REMOTES_FILENAME} has no usable `allowed_push_urls` list of "
"strings. Expected: {\"schema\": 1, \"allowed_push_urls\": [\"<url>\"]}"
)
return None
return urls
def push_url_for(remote: str) -> Optional[str]:
"""The URL `git push <remote>` would actually write to - `pushurl` when the
remote sets one, otherwise its fetch URL. Reading the resolved value rather
than the name is the whole point: a repointed `origin` must not pass."""
result = _run(["git", "remote", "get-url", "--push", remote])
if result.returncode != 0:
return None
return result.stdout.strip() or None
def publish_remote_refusal(remote: str, branch: str) -> Optional[str]:
"""The gate's message when `remote` is not an allowed push target, else None."""
allowed = read_allowed_push_urls()
if allowed is None:
return None
url = push_url_for(remote)
if url is None:
return (
f"Publish-Remote Gate: '{remote}' resolves to no push URL, so this publish "
f"cannot be checked against {config.PUBLISH_REMOTES_FILENAME}. Nothing was "
"committed or pushed."
)
if url in allowed:
return None
listed = "\n".join(f" - {u}" for u in allowed) or " (the list is empty)"
return (
f"Publish-Remote Gate: this checkout does not allow publishing to '{remote}'.\n\n"
f" would push to: {url}\n"
f" allowed here:\n{listed}\n\n"
"Nothing was committed or pushed. This checkout holds content that belongs to it "
"alone, and a push to the wrong remote is not cheaply reversible - the objects stay "
"fetchable by SHA even after a force-push, until the server's reflogs are expired.\n\n"
"THE USER CANNOT SEE THIS OUTPUT. It went to your context, not to their screen.\n"
"Show them the two lines above and stop. There is no flag that opens this gate: if "
f"the target really is right, the user adds its URL to {config.PUBLISH_REMOTES_FILENAME} "
"themselves. Do not edit that file to get past this."
)
def parse_porcelain_entries(stdout: str) -> list[tuple[str, str]]: def parse_porcelain_entries(stdout: str) -> list[tuple[str, str]]:
"""Parse `git status --porcelain -z` output into (status_code, path) pairs. """Parse `git status --porcelain -z` output into (status_code, path) pairs.
@@ -919,6 +999,26 @@ def publish_command(
if push and checked_out != branch: if push and checked_out != branch:
fail(branch_mismatch_message(checked_out, branch)) fail(branch_mismatch_message(checked_out, branch))
# Before the reconcile below, which is the first thing that talks to the
# remote at all: a publish aimed at the wrong repository should not even
# fetch from it, and the refusal message promises that nothing was
# committed or pushed.
if push:
refusal = publish_remote_refusal(remote, branch)
if refusal:
emit(
"wikitool",
"gate.refused",
{
"gate": "publish-remote",
"reason": "remote-not-allowed",
"remote": remote,
"branch": branch,
"url": push_url_for(remote),
},
)
needs_clearance(refusal)
# Pull against the remote before anything else, to minimise the window in which this # Pull against the remote before anything else, to minimise the window in which this
# publish could diverge from it - and, as a side effect, to finally publish a commit left # publish could diverge from it - and, as a side effect, to finally publish a commit left
# stranded by a previous push that failed (see `_local_ahead_of_remote` below). Skipped # stranded by a previous push that failed (see `_local_ahead_of_remote` below). Skipped
+95
View File
@@ -0,0 +1,95 @@
"""`wikitool links` - the declared graph around one page, both directions.
The half that makes authored directional edges liveable. An edge is written once,
on the page that asserts it, so the question "what points at *this* page" has no
answer stored anywhere - it is computed from the graph, which is the only way it
is ever complete. A mirrored edge only ever recorded what someone remembered to
mirror.
Read-only, and exempt from the iteration budget for the same reason `search` is:
it answers a question rather than changing anything, and an agent that has to
ration looking things up starts guessing instead.
"""
from __future__ import annotations
import json as _json
from typing import Optional
import typer
from chemenu import config, links
from chemenu.commands._util import console, fail
from chemenu.kb_scan import load_kb_pages
from chemenu.page import Page
app = typer.Typer(help="Show the declared edges into and out of a page.")
EDGE_FIELD = "related"
def _collection_of(page: Page) -> Optional[str]:
try:
return page.path.relative_to(config.KB_DIR).parts[0]
except (ValueError, IndexError):
return None
def outbound(pages: dict[str, Page], title: str) -> list[dict]:
"""Edges this page asserts, in file order."""
page = pages[title]
return [
{"target": edge.target, "label": edge.label, "resolves": edge.target in pages}
for edge in links.edges(page.frontmatter, EDGE_FIELD)
]
def inbound(pages: dict[str, Page], title: str) -> list[dict]:
"""Edges other pages assert *about* this one.
A full scan of the corpus rather than a stored list, deliberately: the whole
argument for dropping mirrored edges is that this answer is derived and
therefore cannot go stale or be half-written.
"""
found = [
{"source": other, "label": edge.label, "collection": _collection_of(page)}
for other, page in pages.items()
for edge in links.edges(page.frontmatter, EDGE_FIELD)
if edge.target == title
]
return sorted(found, key=lambda item: (item["label"] or "", item["source"]))
@app.command("show")
def links_show(
page: str = typer.Option(..., "--page", help="Exact page title"),
json_out: bool = typer.Option(False, "--json", help="Print the edges as JSON"),
):
"""Show the edges out of and into a page.
Outbound is what the page declares in `related:`. Inbound is computed across
the corpus - nothing stores it, which is exactly why it is complete."""
pages = load_kb_pages(config.KB_DIR)
if page not in pages:
fail(f"No page titled '{page}' found under kb/.")
out, back = outbound(pages, page), inbound(pages, page)
if json_out:
typer.echo(_json.dumps({"page": page, "outbound": out, "inbound": back}, indent=2))
return
console.print(f"[bold]{page}[/bold]")
console.print(f"\n[cyan]asserts ({len(out)})[/cyan]")
if not out:
console.print(" (none)")
for edge in out:
label = edge["label"] or "[dim]unlabelled[/dim]"
missing = "" if edge["resolves"] else " [red](no such page)[/red]"
console.print(f" {label} -> [[{edge['target']}]]{missing}")
console.print(f"\n[cyan]asserted about it ({len(back)})[/cyan]")
if not back:
console.print(" (none - nothing in the corpus declares an edge to this page)")
for edge in back:
label = edge["label"] or "[dim]unlabelled[/dim]"
console.print(f" [[{edge['source']}]] {label} ->")
+27 -415
View File
@@ -1,16 +1,12 @@
"""Deterministic structural health checks for the wiki. """`wikitool lint` - the terminal adapter over `chemenu.lint_core`.
This intentionally covers only what can be computed mechanically: broken The checks, the report and the hard-error rule live in `chemenu/lint_core.py`,
wikilinks, orphan pages, index/page drift, frontmatter schema gaps, and which imports no CLI machinery. This module owns only what a terminal needs:
filename/title mismatches. Semantic judgment (contradictions, staleness, the flags, where the report file lands, and the exit code.
what's worth writing about next) stays with the LLM - this report gives it a
verified factual foundation instead of requiring it to re-derive these facts
by reading every page.
""" """
from __future__ import annotations from __future__ import annotations
import json import json
from datetime import date
from pathlib import Path from pathlib import Path
from typing import Optional from typing import Optional
@@ -18,417 +14,33 @@ import typer
from chemenu import config from chemenu import config
from chemenu.commands._util import rel_path, success from chemenu.commands._util import rel_path, success
from chemenu.frontmatter_io import frontmatter_error from chemenu.lint_core import (
from chemenu.markdown_code import strip_code_spans HARD_ERROR_KEYS,
from chemenu.provenance import broken_raw_refs as find_broken_raw_refs MOST_LINKED_COUNT,
from chemenu.provenance import duplicate_raw_file_owners as find_duplicate_raw_file_owners QUOTE_LIMIT,
from chemenu.provenance import extract_inline_cites count_quote_blocks,
from chemenu.provenance import legacy_citation_markers as find_legacy_citation_markers default_report_path,
from chemenu.provenance import legacy_source_pages as find_legacy_source_pages has_hard_errors,
from chemenu.provenance import orphan_footnote_defs as find_orphan_footnote_defs render_markdown,
from chemenu.provenance import uncovered_raw_files as find_uncovered_raw_files render_summary,
from chemenu.provenance import undefined_footnote_refs as find_undefined_footnote_refs run_lint,
from chemenu.kb_scan import (
GENERATED_INDEX,
WIKILINK_RE,
build_link_graph,
find_duplicate_title_paths,
inbound_links,
load_kb_pages,
) )
from chemenu.type_resolver import resolver
# Style guide's one mechanically-checkable rule (hard oracle: a plain count). # Re-exported: `from chemenu.commands.lint import run_lint` still resolves, and
# The rest of the style guide (tone, AI-phrase avoidance) is a soft/proxy judgment # so does every other name the tests and sibling commands already import.
# and stays with the LLM - see wiki-manage/wiki-ingest skill guidance, not lint. __all__ = [
# "HARD_ERROR_KEYS",
# The unit is a quote, not a `>` line. It used to be the line, which measured "MOST_LINKED_COUNT",
# the wrap width the rule has no opinion about: one quotation written long "QUOTE_LIMIT",
# counted 1 and the same quotation wrapped at 100 columns counted 4. An author "count_quote_blocks",
# who took the finding seriously made the page harder to read to quiet it. "default_report_path",
QUOTE_LIMIT = 2 "has_hard_errors",
"render_markdown",
# How many hub pages `most_linked` reports. Purely informational (wiki-status "render_summary",
# surfaces it); not a finding, so the cutoff only bounds report size. "run_lint",
MOST_LINKED_COUNT = 10 "lint_command",
def count_quote_blocks(body: str) -> int:
"""How many distinct blockquotes `body` carries.
A run of consecutive `>` lines is one quote; a blank line or any
non-quoted line ends it. Code is masked out first, so a `>` inside a
fenced shell transcript is a prompt, not a quotation.
Lazy continuation - a quote whose wrapped lines drop the `>` - reads here
as two quotes rather than one. That over-counts in the direction the limit
already errs on, and the corpus prefixes every line, so the alternative
(tracking paragraph state) buys nothing.
"""
count, in_quote = 0, False
for line in strip_code_spans(body).splitlines():
is_quote = line.lstrip().startswith(">")
if is_quote and not in_quote:
count += 1
in_quote = is_quote
return count
def run_lint(kb_dir: Path) -> dict:
pages = load_kb_pages(kb_dir)
duplicate_titles = find_duplicate_title_paths(kb_dir, config.ROOT)
# Pages whose frontmatter can't be parsed read back as `{}` everywhere
# else, which would let them slip past every frontmatter-driven check
# below with no finding at all - so they are detected explicitly.
frontmatter_errors = []
for title, page in sorted(pages.items()):
reason = frontmatter_error(page.path)
if reason is None and not page.frontmatter.get("type"):
reason = "missing `type:` field"
if reason is not None:
frontmatter_errors.append({"page": title, "error": reason})
graph = build_link_graph(pages)
broken_links = [
{"page": title, "target": target}
for title, targets in graph.items()
for target in sorted(targets)
if target not in pages
] ]
inbound = inbound_links({t: v for t, v in graph.items() if t != "index"})
orphan_pages = sorted(
title
for title, sources in inbound.items()
if not sources
and title not in ("index", "log")
# comparison pages are not linked to by design; index.md is sufficient coverage
and pages[title].kind != "comparison"
)
# Same link graph, opposite end: the most-linked-to pages are the wiki's
# hubs. Reported (not judged) so `wiki-status` can show them without
# re-deriving the graph.
inbound_counts = {title: len(sources) for title, sources in inbound.items()}
most_linked = [
{"page": title, "inbound": count}
for title, count in sorted(inbound_counts.items(), key=lambda kv: (-kv[1], kv[0]))
if count > 0
][:MOST_LINKED_COUNT]
# The catalog is sharded: `kb/index.md` is a map carrying counts and links,
# and the page rows live in a generated INDEX.md per collection/area. Both
# halves have to be read, or every page reads as missing from the index.
index_text = "".join(
path.read_text(encoding="utf-8")
for path in [kb_dir / "index.md", *sorted(kb_dir.rglob(GENERATED_INDEX))]
if path.exists()
)
index_links = {m.group(1).strip() for m in WIKILINK_RE.finditer(index_text)}
missing_from_index = sorted(set(pages) - index_links - {"index", "log"})
dangling_index_entries = sorted(index_links - set(pages))
title_mismatches = []
for title, page in sorted(pages.items()):
if page.kind not in ("entity", "concept"):
continue
h1 = page.h1_title
if h1 is not None and h1 != title:
title_mismatches.append({"page": title, "h1": h1})
unmarked_provenance = []
for title, page in sorted(pages.items()):
if page.kind not in ("entity", "concept"):
continue
sources_list = page.frontmatter.get("sources") or []
if not sources_list and page.frontmatter.get("provenance") != "general":
unmarked_provenance.append(title)
citation_frontmatter_drift = []
for title, page in sorted(pages.items()):
sources_list = set(page.frontmatter.get("sources") or [])
cited = {cited_title for cited_title, _file in extract_inline_cites(page.body)}
cited.discard(title) # a source page citing itself for a specific file within it is not drift
for missing_source in sorted(cited - sources_list):
citation_frontmatter_drift.append({"page": title, "cited_but_not_in_sources": missing_source})
legacy_citation_markers = find_legacy_citation_markers(pages)
undefined_footnote_refs = find_undefined_footnote_refs(pages)
orphan_footnote_defs = find_orphan_footnote_defs(pages)
# The frontmatter half of the link graph. `broken_links` above only walks
# `[[wikilinks]]` in page *bodies*, so a `related:`/`sources:`/`entities:`
# entry naming a page that does not exist - a rename that was not
# propagated, a deleted page, or a URL pasted where a title belongs - used
# to pass every check. Which fields hold page titles is declared by each
# type-spec's `page_ref_fields:`, not hardcoded here.
dangling_frontmatter_refs = []
for title, page in sorted(pages.items()):
type_path = page.frontmatter.get("type")
if not type_path:
continue
try:
ref_fields = resolver.get_page_ref_fields(type_path, page.path)
except ValueError:
continue # unresolvable type is already reported as type_resolution_errors
for field in ref_fields:
for target in page.frontmatter.get(field) or []:
if target not in pages:
dangling_frontmatter_refs.append(
{"page": title, "field": field, "target": target}
)
quote_limit_violations = []
for title, page in sorted(pages.items()):
quote_count = count_quote_blocks(page.body)
if quote_count > QUOTE_LIMIT:
quote_limit_violations.append({"page": title, "quote_count": quote_count})
# Type system validation. Lint reports are not validated here: they are
# written to `reports/` outside kb/ and are never pages, so nothing this
# loop scans can be one.
invalid_type_paths = []
type_resolution_errors = []
schema_validation_errors = []
for title, page in sorted(pages.items()):
type_path = page.frontmatter.get("type")
if not type_path:
continue
# Check if type path is valid
if not type_path.endswith('.md'):
invalid_type_paths.append({"page": title, "type": type_path, "error": "Type path must end with .md"})
continue
# Try to resolve and validate the type
try:
resolver.load_type_spec(type_path, page.path)
# Try schema validation
try:
resolver.validate_frontmatter(page.frontmatter, type_path, page.path)
except ValueError as schema_error:
schema_validation_errors.append({"page": title, "type": type_path, "error": str(schema_error)})
except ValueError as resolution_error:
type_resolution_errors.append({"page": title, "type": type_path, "error": str(resolution_error)})
return {
"generated": date.today().isoformat(),
"page_count": len(pages),
"frontmatter_errors": frontmatter_errors,
"broken_links": broken_links,
"orphan_pages": orphan_pages,
"most_linked": most_linked,
"inbound_counts": inbound_counts,
"missing_from_index": missing_from_index,
"dangling_index_entries": dangling_index_entries,
"title_mismatches": title_mismatches,
"duplicate_titles": duplicate_titles,
"uncovered_raw_files": find_uncovered_raw_files(config.RAW_DIR, pages),
"broken_raw_refs": find_broken_raw_refs(pages),
"duplicate_raw_file_owners": find_duplicate_raw_file_owners(pages),
"legacy_source_pages": find_legacy_source_pages(pages),
"unmarked_provenance": unmarked_provenance,
"citation_frontmatter_drift": citation_frontmatter_drift,
"legacy_citation_markers": legacy_citation_markers,
"undefined_footnote_refs": undefined_footnote_refs,
"orphan_footnote_defs": orphan_footnote_defs,
"dangling_frontmatter_refs": dangling_frontmatter_refs,
"quote_limit_violations": quote_limit_violations,
"invalid_type_paths": invalid_type_paths,
"type_resolution_errors": type_resolution_errors,
"schema_validation_errors": schema_validation_errors,
}
def _section(lines: list[str], title: str, items: list, formatter) -> None:
lines.append(f"## {title}")
lines.append("")
if not items:
lines.append("None found.")
else:
for item in items:
lines.append(f"- {formatter(item)}")
lines.append("")
def render_markdown(report: dict) -> str:
lines = [f"# Structural Lint Report ({report['generated']})", ""]
lines.append(f"Scanned {report['page_count']} pages under `wiki/`. This report covers only")
lines.append("mechanically-verifiable structural issues; see the Semantic Review section")
lines.append("below for judgment calls the LLM should complete.")
lines.append("")
_section(
lines, "Unreadable Frontmatter", report["frontmatter_errors"],
lambda i: f"[[{i['page']}]] - {i['error']}",
)
_section(
lines, "Broken Wikilinks", report["broken_links"],
lambda i: f"[[{i['page']}]] links to missing [[{i['target']}]]",
)
_section(lines, "Orphan Pages (no inbound links)", report["orphan_pages"], lambda i: f"[[{i}]]")
_section(
lines, f"Most-Linked Pages (top {MOST_LINKED_COUNT} hubs)", report["most_linked"],
lambda i: f"[[{i['page']}]] - {i['inbound']} inbound link(s)",
)
_section(lines, "Pages Missing from index.md", report["missing_from_index"], lambda i: f"[[{i}]]")
_section(lines, "Dangling index.md Entries", report["dangling_index_entries"], lambda i: f"[[{i}]]")
_section(
lines, "Duplicate Titles (naming collisions)", report["duplicate_titles"],
lambda i: f"`{i['stem']}` -> {', '.join(f'`{p}`' for p in i['paths'])}",
)
_section(
lines, "Filename / H1 Title Mismatches", report["title_mismatches"],
lambda i: f"[[{i['page']}]] H1 is '{i['h1']}'",
)
_section(
lines, "Uncovered Raw Files (no source page)", report["uncovered_raw_files"],
lambda i: f"`{i}`",
)
_section(
lines, "Broken raw_files References", report["broken_raw_refs"],
lambda i: f"[[{i['page']}]] -> `{i['raw_path']}` (does not exist)",
)
_section(
lines, "Raw Files With More Than One Owner", report["duplicate_raw_file_owners"],
lambda i: f"`{i['raw_file']}` is claimed by " + ", ".join(f"[[{t}]]" for t in i["owners"]),
)
_section(
lines, "Legacy source: Field (not yet migrated to raw_files:)", report["legacy_source_pages"],
lambda i: f"[[{i['page']}]] source: `{i['source']}` ({i['reason']})",
)
_section(
lines, "Pages Missing provenance: general Marker", report["unmarked_provenance"],
lambda i: f"[[{i}]] has no sources and is not marked `provenance: general`",
)
_section(
lines, "Citation / Frontmatter Drift", report["citation_frontmatter_drift"],
lambda i: f"[[{i['page']}]] cites [[{i['cited_but_not_in_sources']}]] inline but it is missing from frontmatter `sources:`",
)
_section(
lines, "Legacy Citation Markers (pre-migration `^[[...]]`)", report["legacy_citation_markers"],
lambda i: f"[[{i['page']}]] still has `{i['marker']}` - run `wikitool cite add` and replace it with the `[^cite-id]` it prints",
)
_section(
lines, "Undefined Footnote References", report["undefined_footnote_refs"],
lambda i: f"[[{i['page']}]] references `[^{i['ref']}]`, which has no `[^{i['ref']}]: [[...]]` definition",
)
_section(
lines, "Orphan Footnote Definitions", report["orphan_footnote_defs"],
lambda i: f"[[{i['page']}]] defines `[^{i['id']}]` (-> [[{i['source']}]]) but nothing references it - run `wikitool cite sync`",
)
_section(
lines, "Dangling Frontmatter References", report["dangling_frontmatter_refs"],
lambda i: f"[[{i['page']}]] `{i['field']}:` names `{i['target']}`, which is not a page",
)
_section(
lines, "Invalid Type Paths", report["invalid_type_paths"],
lambda i: f"[[{i['page']}]] has type: `{i['type']}` - {i['error']}",
)
_section(
lines, "Type Resolution Errors", report["type_resolution_errors"],
lambda i: f"[[{i['page']}]] type: `{i['type']}` - {i['error']}",
)
_section(
lines, "Schema Validation Errors", report["schema_validation_errors"],
lambda i: f"[[{i['page']}]] type: `{i['type']}` - {i['error']}",
)
_section(
lines, f"Pages Exceeding Quote Limit (>{QUOTE_LIMIT}/page)", report["quote_limit_violations"],
lambda i: f"[[{i['page']}]] has {i['quote_count']} quotes - trim or confirm they're load-bearing",
)
lines.append("## Semantic Review (LLM to complete)")
lines.append("")
lines.append("- Contradictions across pages: TODO")
lines.append("- Stale claims (unconfirmed >6 months): TODO")
lines.append("- Suggested new pages / missing cross-references: TODO")
lines.append("")
return "\n".join(lines)
# Sections that always carry content but are not findings, so the summary
# handles them separately: a hub list is a statistic, and the semantic review
# is the checklist that follows the report rather than part of it.
INFORMATIONAL_SECTIONS = ("Most-Linked Pages",)
SEMANTIC_REVIEW_SECTION = "Semantic Review"
def _split_sections(markdown: str) -> tuple[str, list[tuple[str, str]]]:
"""Cut a rendered report into its preamble and (title, body) sections."""
preamble, *rest = markdown.split("\n## ")
sections = []
for part in rest:
title, _, body = part.partition("\n")
sections.append((title.strip(), body.strip()))
return preamble.rstrip(), sections
def render_summary(report: dict) -> str:
"""The same report with the empty sections removed.
On a healthy corpus the full report is better than 90% "None found.", so
reading it in the terminal means paging past the answer. The file on disk
stays complete - this is what gets printed, and the written path underneath
it is how the rest is reached without running lint a second time.
"""
preamble, sections = _split_sections(render_markdown(report))
findings, trailing = [], []
for title, body in sections:
if title.startswith(SEMANTIC_REVIEW_SECTION):
trailing.append((title, body))
elif body != "None found." and not title.startswith(INFORMATIONAL_SECTIONS):
findings.append((title, body))
lines = [preamble, ""]
if not findings:
lines += ["No structural findings.", ""]
for title, body in findings + trailing:
lines += [f"## {title}", "", body, ""]
return "\n".join(lines)
def default_report_path(report: dict) -> Path:
"""Where a report goes when the caller names no path.
`reports/` is derived and gitignored ([reports/CONTRACT.md]), so writing
here by default costs the tree nothing.
"""
return config.ROOT / "reports" / f"Lint Report {report['generated']}.md"
# Findings that make a tree structurally wrong rather than merely untidy.
# `orphan_pages` is deliberately absent: many pages are validly reachable
# through the index or navigation only. `quote_limit_violations` is advisory
# too - it flags a habit, not a broken tree.
#
# One definition, used by `lint --fail-on-error` and by the eval scorecard: if
# the two disagreed, a run could pass its score while lint refused it.
HARD_ERROR_KEYS = (
"frontmatter_errors",
"broken_links",
"dangling_index_entries",
"duplicate_titles",
"broken_raw_refs",
"duplicate_raw_file_owners",
"legacy_source_pages",
"citation_frontmatter_drift",
"legacy_citation_markers",
"undefined_footnote_refs",
"orphan_footnote_defs",
"dangling_frontmatter_refs",
"invalid_type_paths",
"type_resolution_errors",
"schema_validation_errors",
)
def has_hard_errors(report: dict) -> bool:
return any(report.get(key) for key in HARD_ERROR_KEYS)
def lint_command( def lint_command(
json_out: bool = typer.Option(False, "--json", help="Print the raw findings as JSON and write no report"), json_out: bool = typer.Option(False, "--json", help="Print the raw findings as JSON and write no report"),
+91 -3
View File
@@ -64,6 +64,7 @@ def list_command(
"name": m.name, "name": m.name,
"migrates_to": str(m.target), "migrates_to": str(m.target),
"migration_kind": m.kind, "migration_kind": m.kind,
"obligation": m.obligation,
"description": m.description, "description": m.description,
"path": m.relative_path, "path": m.relative_path,
} }
@@ -78,7 +79,10 @@ def list_command(
success(f"No migration documents under {rel_path(kb_state.migrations_dir())}.") success(f"No migration documents under {rel_path(kb_state.migrations_dir())}.")
return return
for migration in migrations: for migration in migrations:
console.print(f"[bold]{migration.target}[/bold] {migration.name} ({migration.kind})") console.print(
f"[bold]{migration.target}[/bold] {migration.name} "
f"({migration.kind}, {migration.obligation})"
)
if migration.description: if migration.description:
console.print(f" {migration.description}") console.print(f" {migration.description}")
@@ -86,6 +90,50 @@ def list_command(
# --- migrate status -------------------------------------------------------- # --- migrate status --------------------------------------------------------
def _report_offers(
offered: list["kb_state.Migration"], divergent: Optional[list[str]]
) -> None:
"""Print the optional half of `status`, above the outstanding chain.
Deliberately never affects the exit code and never says "outstanding". An
offer is the stack proposing a better default for a file the instance owns;
an instance that keeps its own version is in a correct state, not a late
one. Mixing the two is how the message that actually matters - your content
no longer fits your machinery - stops being read.
"""
if not offered:
return
console.print(
f"[cyan]{len(offered)} optional upgrade(s) available[/cyan] - none of them block:"
)
for migration in offered:
console.print(f" {migration.target} {migration.name} ({migration.kind})")
if migration.description:
console.print(f" {migration.description}")
console.print(f" {migration.relative_path}")
if divergent is None:
console.print(
" [dim]This tree carries no release stamp, so which of your files still match "
"what you were given cannot be answered here.[/dim]"
)
return
if divergent:
console.print(
f" [dim]{len(divergent)} file(s) differ from the release you installed - those are "
"yours to reconcile by hand rather than overwrite:[/dim]"
)
for relative in divergent[:10]:
console.print(f" [dim]{relative}[/dim]")
if len(divergent) > 10:
console.print(f" [dim]... and {len(divergent) - 10} more[/dim]")
else:
console.print(
" [dim]No file differs from the release you installed, so an offer can be taken "
"by copying.[/dim]"
)
@app.command("status") @app.command("status")
def status_command( def status_command(
json_out: bool = typer.Option(False, "--json", help="Print the chain as JSON"), json_out: bool = typer.Option(False, "--json", help="Print the chain as JSON"),
@@ -113,6 +161,8 @@ def status_command(
return return
pending = kb_state.chain(migrations, kb_version, stack) pending = kb_state.chain(migrations, kb_version, stack)
offered = kb_state.offers(migrations, kb_state.applied_names(kb_state.read_kb_state()))
divergent = kb_state.divergent_files()
if json_out: if json_out:
typer.echo( typer.echo(
@@ -124,6 +174,11 @@ def status_command(
{"name": m.name, "migrates_to": str(m.target), "migration_kind": m.kind} {"name": m.name, "migrates_to": str(m.target), "migration_kind": m.kind}
for m in pending for m in pending
], ],
"offered": [
{"name": m.name, "migrates_to": str(m.target), "migration_kind": m.kind}
for m in offered
],
"divergent_files": divergent,
}, },
indent=2, indent=2,
) )
@@ -131,6 +186,7 @@ def status_command(
return return
console.print(f"stack {stack}, content {kb_version}") console.print(f"stack {stack}, content {kb_version}")
_report_offers(offered, divergent)
if not pending: if not pending:
if kb_version < stack: if kb_version < stack:
console.print( console.print(
@@ -166,7 +222,12 @@ def done_command(
Refuses any version that is not the *next* link in the chain: skipping a Refuses any version that is not the *next* link in the chain: skipping a
migration is how a corpus ends up in a shape no version describes, and an migration is how a corpus ends up in a shape no version describes, and an
interrupted multi-step upgrade has to be resumable rather than guessable.""" interrupted multi-step upgrade has to be resumable rather than guessable.
An `offered` migration is recorded but does not move the version, and no
ordering rule applies to it - it is not a link in the chain. The record is
the only thing that distinguishes an offer someone took from one they
ignored, precisely because the version stays put."""
stack, kb_version = _versions() stack, kb_version = _versions()
if kb_version is None: if kb_version is None:
fail( fail(
@@ -182,6 +243,34 @@ def done_command(
return return
migrations = kb_state.load_migrations() migrations = kb_state.load_migrations()
state = kb_state.read_kb_state() or {}
# An offer is recorded but does not advance the version: it is not a link in
# the chain, so there is no ordering rule to check and nothing to skip. The
# ledger is what makes it stop being offered - without that record there
# would be no way to tell a taken offer from an ignored one, because
# `kb_version` deliberately does not move.
offered = {m.name: m for m in migrations if not m.is_required}
taken = next((m for m in offered.values() if str(m.target) == version), None)
if taken is not None:
if taken.name in kb_state.applied_names(state):
success(f"{taken.name} is already recorded as taken. Nothing to do.")
return
if dry_run:
success(f"Dry run: would record the optional {taken.name}. Nothing written.")
return
applied = list(state.get("applied") or [])
entry = {"migration": taken.name, "at": today_iso(), "obligation": kb_state.OFFERED}
if pages is not None:
entry["pages"] = pages
applied.append(entry)
kb_state.write_kb_state(kb_version, applied)
success(
f"Recorded the optional {taken.name}. Content stays at {kb_version} - an offer "
"changes a file you own, not the shape of your content."
)
return
expected = kb_state.next_link(migrations, kb_version, stack) expected = kb_state.next_link(migrations, kb_version, stack)
if expected is None: if expected is None:
fail( fail(
@@ -197,7 +286,6 @@ def done_command(
) )
return return
state = kb_state.read_kb_state() or {}
applied = list(state.get("applied") or []) applied = list(state.get("applied") or [])
entry = {"migration": expected.name, "at": today_iso()} entry = {"migration": expected.name, "at": today_iso()}
if pages is not None: if pages is not None:
+6 -1
View File
@@ -324,7 +324,12 @@ def new_page_command(
path = target_dir / f"{page_title}.md" path = target_dir / f"{page_title}.md"
body = _apply_template_variables( body = _apply_template_variables(
template, {**frontmatter, "name": name, "today": today.isoformat()} template,
{
**frontmatter,
"name": name,
"today": today.isoformat(),
},
) )
write_page(path, frontmatter, body) write_page(path, frontmatter, body)
+11 -8
View File
@@ -25,7 +25,7 @@ from typing import Optional
import typer import typer
from chemenu import config from chemenu import config, links
from chemenu.commands._util import check_collision, fail, rel_path, success from chemenu.commands._util import check_collision, fail, rel_path, success
from chemenu.frontmatter_io import write_page from chemenu.frontmatter_io import write_page
from chemenu.page import Page from chemenu.page import Page
@@ -33,7 +33,6 @@ from chemenu.kb_scan import load_kb_pages
from chemenu.provenance import ( from chemenu.provenance import (
CITE_REF_RE, CITE_REF_RE,
cite_id, cite_id,
cite_block_heading,
render_page_body, render_page_body,
split_cite_block, split_cite_block,
unique_cite_id, unique_cite_id,
@@ -103,7 +102,7 @@ def retarget_cite_ids(body: str, old: str, new: str) -> str:
return body return body
new_head = CITE_REF_RE.sub(lambda m: f"[^{renames.get(m.group(1), m.group(1))}]", head) new_head = CITE_REF_RE.sub(lambda m: f"[^{renames.get(m.group(1), m.group(1))}]", head)
return render_page_body(new_head, new_definitions, cite_block_heading(body)) return render_page_body(new_head, new_definitions)
def retarget_frontmatter(page: Page, old: str, new: str) -> bool: def retarget_frontmatter(page: Page, old: str, new: str) -> bool:
@@ -114,9 +113,11 @@ def retarget_frontmatter(page: Page, old: str, new: str) -> bool:
values = page.frontmatter.get(field) values = page.frontmatter.get(field)
if not values: if not values:
continue continue
updated = [new if value == old else value for value in values] # Through `links` so a labelled edge keeps its label across a rename:
if updated != values: # the entry is `{label: target}`, and a plain equality swap would have
page.frontmatter[field] = updated # compared the mapping against a title and silently left it pointing at
# the old page.
if links.retarget(page.frontmatter, field, old, new):
changed = True changed = True
return changed return changed
@@ -157,8 +158,10 @@ def strip_frontmatter_ref(page: Page, title: str) -> bool:
values = page.frontmatter.get(field) values = page.frontmatter.get(field)
if not values: if not values:
continue continue
updated = [value for value in values if value != title] before = list(values)
if updated == values: links.remove(page.frontmatter, field, title)
updated = page.frontmatter.get(field) or []
if updated == before:
continue continue
if not updated and field not in declared: if not updated and field not in declared:
del page.frontmatter[field] del page.frontmatter[field]
+4
View File
@@ -82,6 +82,10 @@ SKIP_COMMAND_PATHS = {
("eval", "score"), ("eval", "score"),
("eval", "sessions"), ("eval", "sessions"),
("cite", "id"), ("cite", "id"),
# Retrieval, like `search`: an agent that has to ration looking up what
# points at a page starts guessing instead - and under authored directional
# edges this is the *only* way to ask that question.
("links", "show"),
("version", "show"), ("version", "show"),
("version", "check"), ("version", "check"),
("version", "notes"), ("version", "notes"),
+30 -71
View File
@@ -21,90 +21,38 @@ Scope is `kb/` only. `instructions/` is discovered through
from __future__ import annotations from __future__ import annotations
import json import json
from pathlib import Path
import typer import typer
from chemenu import config
from chemenu.commands._util import fail, today_iso from chemenu.commands._util import fail, today_iso
from chemenu.frontmatter_io import read_page
from chemenu.kb_scan import iter_kb_pages
from chemenu.page import Page
from chemenu.search import filters from chemenu.search import filters
from chemenu.search.base import page_key
from chemenu.search.filters import PredicateError from chemenu.search.filters import PredicateError
from chemenu.search.fuse import reciprocal_rank_fusion
from chemenu.search.registry import UnknownBackend, resolve from chemenu.search.registry import UnknownBackend, resolve
from chemenu.search.ripgrep import RipgrepFailed, RipgrepMissing, build_hit from chemenu.search.ripgrep import RipgrepFailed, RipgrepMissing
from chemenu.search.service import (
load_pages_by_path,
run_search,
sort_hits,
unreadable_pages,
)
from chemenu.search.types import Predicate, SearchHit, SearchQuery from chemenu.search.types import Predicate, SearchHit, SearchQuery
# Re-exported so `from chemenu.commands.search import run_search` keeps
# resolving. The core lives in `chemenu/search/service.py`, which imports no
# CLI machinery; this module is the terminal adapter over it.
__all__ = [
"load_pages_by_path",
"run_search",
"sort_hits",
"unreadable_pages",
"render_table",
"search_command",
]
TITLE_WIDTH = 34 TITLE_WIDTH = 34
SUMMARY_WIDTH = 84 SUMMARY_WIDTH = 84
def load_pages_by_path(kb_dir: Path | None = None, root: Path | None = None) -> dict[str, Page]:
"""Every page under `kb/`, keyed by repo-relative path.
Path-keyed rather than title-keyed on purpose: `load_kb_pages()` drops one
of two pages sharing a stem, and search should still find both - a
duplicate title is a lint finding, not a reason to hide a page.
"""
kb_dir = kb_dir or config.KB_DIR
root = root or config.ROOT
pages: dict[str, Page] = {}
for path in iter_kb_pages(kb_dir):
frontmatter, body = read_page(path)
pages[page_key(path, root)] = Page(path=path, frontmatter=frontmatter, body=body)
return pages
def _sort_key(hit: SearchHit, field: str):
value = hit.as_dict().get(field)
if value is None:
# Missing values sort last in either direction rather than crashing on
# a None comparison.
return (1, "")
if isinstance(value, (int, float)):
return (0, value)
return (0, str(value).lower())
def sort_hits(hits: list[SearchHit], sort: str | None) -> list[SearchHit]:
"""Sort by a hit field. A leading `-` reverses, e.g. `--sort -confidence`."""
if not sort:
return hits
descending = sort.startswith("-")
field = sort.lstrip("-")
ordered = sorted(hits, key=lambda h: _sort_key(h, field), reverse=descending)
return ordered
def run_search(
query: SearchQuery,
pages: dict[str, Page],
backends: list,
kb_dir: Path | None = None,
) -> list[SearchHit]:
"""Answer a query. Pure: no I/O beyond whatever a backend does."""
filters.validate_fields(query.predicates, pages)
if query.text:
rankings = [backend.search(query, pages) for backend in backends]
hits = rankings[0] if len(rankings) == 1 else reciprocal_rank_fusion(rankings)
allowed = filters.apply_predicates(pages, query.predicates, kb_dir)
hits = [hit for hit in hits if hit.path in allowed]
else:
selected = filters.apply_predicates(pages, query.predicates, kb_dir)
hits = [
build_hit(page, key, [], query, backend="frontmatter", kb_dir=kb_dir)
for key, page in selected.items()
]
hits.sort(key=lambda h: h.title.lower())
hits = sort_hits(hits, query.sort)
return hits[: query.limit] if query.limit else hits
def _truncate(text: str, width: int) -> str: def _truncate(text: str, width: int) -> str:
text = " ".join(text.split()) text = " ".join(text.split())
return text if len(text) <= width else text[: width - 1] + "\u2026" return text if len(text) <= width else text[: width - 1] + "\u2026"
@@ -206,6 +154,8 @@ def search_command(
except RipgrepFailed as exc: except RipgrepFailed as exc:
fail(str(exc)) fail(str(exc))
unreadable = unreadable_pages(pages)
if json_out: if json_out:
payload = { payload = {
"generated": today_iso(), "generated": today_iso(),
@@ -214,8 +164,17 @@ def search_command(
"backend": ",".join(b.name for b in backends), "backend": ",".join(b.name for b in backends),
"count": len(hits), "count": len(hits),
"results": [hit.as_dict() for hit in hits], "results": [hit.as_dict() for hit in hits],
# Always present, usually empty. A caller that has to look for the
# key to learn whether it should worry will not look.
"unreadable": unreadable,
} }
typer.echo(json.dumps(payload, indent=2)) typer.echo(json.dumps(payload, indent=2))
return return
typer.echo(render_table(hits, show_matches)) typer.echo(render_table(hits, show_matches))
for entry in unreadable:
typer.echo(
f"WARN unreadable frontmatter: {entry['path']} ({entry['reason']}) - "
"this page cannot match any --field predicate",
err=True,
)
+25 -56
View File
@@ -7,35 +7,29 @@ into context on every skill invocation. A type-spec's own frontmatter
declared `.schema.yaml` are the single source of truth; this command only declared `.schema.yaml` are the single source of truth; this command only
formats what `TypeResolver` already resolves - it does not duplicate or formats what `TypeResolver` already resolves - it does not duplicate or
re-derive any type knowledge. re-derive any type knowledge.
The resolving half lives in `chemenu/types_core.py`, which imports no CLI
machinery. This module is the terminal adapter over it.
""" """
from __future__ import annotations from __future__ import annotations
import json import json
from typing import Any, Dict
import typer import typer
from chemenu.commands._util import fail from chemenu.commands._util import fail
from chemenu.type_resolver import resolver from chemenu.types_core import UnknownType, describe_type, list_types
app = typer.Typer(help="Discover and describe Chemenu type-spec contracts.") app = typer.Typer(help="Discover and describe Chemenu type-spec contracts.")
@app.command("list") @app.command("list")
def list_types(json_out: bool = typer.Option(False, "--json", help="Print raw findings as JSON")): def list_types_command(
json_out: bool = typer.Option(False, "--json", help="Print raw findings as JSON")
):
"""List every type-spec under types/, with its name, schema, subtype """List every type-spec under types/, with its name, schema, subtype
field (if any), base directory, and description.""" field (if any), base directory, and description."""
rows: list[Dict[str, Any]] = [] rows = list_types()
for type_path, frontmatter in resolver.list_type_specs():
rows.append({
"name": frontmatter.get("name"),
"type_path": type_path,
"schema": frontmatter.get("schema"),
"subtype_field": frontmatter.get("subtype_field"),
"root": frontmatter.get("root") or "kb",
"base_dir": frontmatter.get("base_dir"),
"description": frontmatter.get("description"),
})
if json_out: if json_out:
typer.echo(json.dumps(rows, indent=2)) typer.echo(json.dumps(rows, indent=2))
@@ -53,7 +47,7 @@ def list_types(json_out: bool = typer.Option(False, "--json", help="Print raw fi
@app.command("describe") @app.command("describe")
def describe_type( def describe_type_command(
name: str = typer.Argument(..., help="Type name, e.g. 'entity' (see `types list`)"), name: str = typer.Argument(..., help="Type name, e.g. 'entity' (see `types list`)"),
json_out: bool = typer.Option(False, "--json", help="Print raw findings as JSON"), json_out: bool = typer.Option(False, "--json", help="Print raw findings as JSON"),
): ):
@@ -61,51 +55,26 @@ def describe_type(
with enums where declared), its subtype field if any, and its authoring with enums where declared), its subtype field if any, and its authoring
body - the same information an LLM would otherwise gather by reading the body - the same information an LLM would otherwise gather by reading the
raw type-spec and `.schema.yaml` files directly.""" raw type-spec and `.schema.yaml` files directly."""
type_path = resolver.find_type_by_name(name) try:
if type_path is None: described = describe_type(name)
available = sorted(fm.get("name") for _, fm in resolver.list_type_specs()) except UnknownType as exc:
fail(f"No type-spec named '{name}'. Available: {', '.join(available)}") fail(str(exc))
return # unreachable; keeps type-checkers happy about `type_path` below return # unreachable; keeps type-checkers happy about `described` below
type_spec = resolver.load_type_spec(type_path)
frontmatter = type_spec["frontmatter"]
body = type_spec["body"]
schema = resolver.get_schema(type_path)
fields: list[Dict[str, Any]] = []
if schema is not None:
required = set(schema.get("required", []))
for field_name, field_schema in schema.get("properties", {}).items():
fields.append({
"field": field_name,
"required": field_name in required,
"type": field_schema.get("type"),
"enum": field_schema.get("enum"),
})
if json_out: if json_out:
typer.echo(json.dumps({ typer.echo(json.dumps(described, indent=2))
"name": frontmatter.get("name"),
"type_path": type_path,
"description": frontmatter.get("description"),
"schema": frontmatter.get("schema"),
"subtype_field": frontmatter.get("subtype_field"),
"base_dir": frontmatter.get("base_dir"),
"title_prefix": frontmatter.get("title_prefix"),
"fields": fields,
"body": body.strip(),
}, indent=2))
return return
typer.echo(f"# {frontmatter.get('name')} ({type_path})") fields = described["fields"]
typer.echo(frontmatter.get("description", "")) typer.echo(f"# {described['name']} ({described['type_path']})")
typer.echo(described["description"] or "")
typer.echo("") typer.echo("")
if frontmatter.get("subtype_field"): if described["subtype_field"]:
typer.echo(f"subtype_field: {frontmatter['subtype_field']}") typer.echo(f"subtype_field: {described['subtype_field']}")
if frontmatter.get("base_dir"): if described["base_dir"]:
typer.echo(f"base_dir: {frontmatter.get('root') or 'kb'}/{frontmatter['base_dir']}") typer.echo(f"base_dir: {described['root']}/{described['base_dir']}")
if frontmatter.get("title_prefix"): if described["title_prefix"]:
typer.echo(f"title_prefix: {frontmatter['title_prefix']!r}") typer.echo(f"title_prefix: {described['title_prefix']!r}")
typer.echo("") typer.echo("")
if not fields: if not fields:
@@ -119,4 +88,4 @@ def describe_type(
typer.echo("") typer.echo("")
typer.echo("## Authoring guidance") typer.echo("## Authoring guidance")
typer.echo(body.strip()) typer.echo(described["body"])
+31 -4
View File
@@ -189,6 +189,11 @@ def bump_command(
minor: bool = typer.Option(False, "--minor", help="Bump MINOR (resets PATCH)"), minor: bool = typer.Option(False, "--minor", help="Bump MINOR (resets PATCH)"),
patch: bool = typer.Option(False, "--patch", help="Bump PATCH"), patch: bool = typer.Option(False, "--patch", help="Bump PATCH"),
title: str = typer.Option(..., "--title", help="One-line title for the new CHANGES.md entry"), title: str = typer.Option(..., "--title", help="One-line title for the new CHANGES.md entry"),
breaking: Optional[str] = typer.Option(
None,
"--breaking",
help="What stops working, for a boundary-crossing bump (recorded in CHANGES.md). Required on one, refused on any other",
),
no_migration: Optional[str] = typer.Option( no_migration: Optional[str] = typer.Option(
None, None,
"--no-migration", "--no-migration",
@@ -203,10 +208,14 @@ def bump_command(
frontmatter and leaves the prose. `docs verify` afterwards enforces that frontmatter and leaves the prose. `docs verify` afterwards enforces that
the two agree, so a bump with no entry cannot reach a release. the two agree, so a bump with no entry cannot reach a release.
A bump that crosses the compatibility boundary additionally requires a A bump that crosses the compatibility boundary - one whose new version is
migration document for the new version, or `--no-migration "<reason>"`. not a drop-in replacement, whether or not any content moves - requires
An instance learning that it must migrate, with nothing telling it how, is `--breaking "<what stops working>"`, and on top of that either a migration
the gap this closes.""" document for the new version or `--no-migration "<reason>"`. An instance
learning that it must migrate, with nothing telling it what broke or how to
cross, is the gap these close. Which part to pass stays a judgment call
this command does not make - it enforces only that a crossing says what it
costs."""
selected = [name for name, chosen in (("major", major), ("minor", minor), ("patch", patch)) if chosen] selected = [name for name, chosen in (("major", major), ("minor", minor), ("patch", patch)) if chosen]
if len(selected) != 1: if len(selected) != 1:
fail("Pass exactly one of --major / --minor / --patch") fail("Pass exactly one of --major / --minor / --patch")
@@ -239,6 +248,23 @@ def bump_command(
crossing = new_version.compat_key != current.compat_key crossing = new_version.compat_key != current.compat_key
boundary = " (crosses a compatibility boundary - instances must migrate)" if crossing else "" boundary = " (crosses a compatibility boundary - instances must migrate)" if crossing else ""
if crossing and not breaking:
fail(
f"{current} -> {new_version} crosses the compatibility boundary, so it is not a "
f"drop-in replacement - re-run with --breaking \"<what stops working, and what an "
f"instance must do about it>\".\n"
f"If that sentence is hard to write because nothing actually breaks - no hand-work "
f"on update, and the old version can still be put back - then the bump is probably "
f"not --{selected[0]}."
)
return
if breaking and not crossing:
fail(
f"--breaking only applies to a bump that crosses the compatibility boundary; "
f"{current} -> {new_version} does not."
)
return
if crossing and not no_migration: if crossing and not no_migration:
from chemenu import kb_state from chemenu import kb_state
@@ -267,6 +293,7 @@ def bump_command(
version_mod.insert_changes_entry( version_mod.insert_changes_entry(
text, new_version, today_iso(), title.strip(), author, text, new_version, today_iso(), title.strip(), author,
no_migration_reason=no_migration.strip() if no_migration else None, no_migration_reason=no_migration.strip() if no_migration else None,
breaking_reason=breaking.strip() if breaking else None,
), ),
encoding="utf-8", encoding="utf-8",
) )
+116 -96
View File
@@ -1,9 +1,22 @@
"""Bidirectional cross-reference management between wiki pages. """Cross-reference management between wiki pages.
`xref add` keeps two pages' frontmatter `related:` lists AND their body `xref add` writes **one** edge: a label plus a target, into the asserting page's
"## Relationships" sections in sync in one operation, instead of the 3-5 `related:` frontmatter, and re-renders that page's generated links region from
separate manual edits this used to take per pair of pages. It is idempotent: it. It is idempotent, and re-running with a different label relabels rather than
re-running it never duplicates a link. duplicating.
It used to write four things at once - `related:` and a Relationships bullet on
both pages, plus reciprocal See Also bullets. That made every edge symmetric by
construction, which is not what a link means: an edge is an authored reader aid,
and "follow this to verify the premise" rarely reads the same from the other
end. Worse, it is incompatible with per-collection label authorisation, because
the mirrored half is written into a collection whose rules the author never
read.
The reverse direction is therefore authored separately, when it is a primary
statement of its own - and navigation does not depend on anyone bothering:
`index rebuild` renders the inbound view from the graph, completely and without
maintenance. See instructions/link-taxonomy.md.
""" """
from __future__ import annotations from __future__ import annotations
@@ -12,7 +25,7 @@ from pathlib import Path
import typer import typer
from chemenu import config, sections from chemenu import blocks, config, conventions, kb_collections, links
from chemenu.commands._util import fail, parse_list, success from chemenu.commands._util import fail, parse_list, success
from chemenu.commands.page_ops import strip_frontmatter_ref from chemenu.commands.page_ops import strip_frontmatter_ref
from chemenu.frontmatter_io import write_page from chemenu.frontmatter_io import write_page
@@ -71,121 +84,119 @@ def _back_reference_field(source: Page, target: Page) -> str | None:
return collection if collection in _declared_ref_fields(source) else None return collection if collection in _declared_ref_fields(source) else None
def add_related(frontmatter: dict, other_title: str) -> bool: def _collection_of(page: Page) -> str | None:
"""Add other_title to frontmatter['related'] if not already present. """The collection a page lives in, or None if it is outside `kb/`."""
Returns True if a change was made.""" try:
related = frontmatter.setdefault("related", []) return page.path.relative_to(config.KB_DIR).parts[0]
if other_title in related: except (ValueError, IndexError):
return False
related.append(other_title)
return True
def _section_bounds(body: str, heading: str) -> tuple[int, int] | None:
match = sections.heading_re(heading).search(body)
if not match:
return None return None
start = match.end()
next_heading = re.search(r"^## ", body[start:], re.MULTILINE)
end = start + next_heading.start() if next_heading else len(body)
return start, end
def add_bullet_to_section(body: str, heading: str, bullet: str, dedup_link: str) -> str: def render_links_block(page: Page) -> str:
"""Insert `bullet` into the `## {heading}` section of body, unless a """The page's generated links region, built from its `related:` edges.
wikilink to dedup_link already appears there. Creates the section
(before the See Also section if present, else at the end) if missing.
`heading` is a canonical name from `sections`; an existing section is found The body is a *rendering* of the frontmatter, not a second place the graph
under its aliases too, so a page that has not been translated yet is still is stored. That is what removed the need to parse a German bullet back into
appended to rather than given a duplicate section. A section this creates a relationship: the label lives in the data, and this writes it out.
always carries the canonical name.""" """
bounds = _section_bounds(body, heading) lines = []
if bounds is None: for edge in links.edges(page.frontmatter, "related"):
section = f"## {heading}\n\n{bullet}\n\n" if edge.is_labelled:
see_also = sections.heading_re(sections.SEE_ALSO).search(body) lines.append(f"- **{edge.label}:** [[{edge.target}]]")
if heading != sections.SEE_ALSO and see_also: else:
return body[: see_also.start()] + section + body[see_also.start() :] lines.append(f"- [[{edge.target}]]")
return body.rstrip("\n") + "\n\n" + section.rstrip("\n") + "\n" return blocks.render(blocks.LINKS, conventions.heading(blocks.LINKS), lines)
start, end = bounds
section_text = body[start:end]
if f"[[{dedup_link}]]" in section_text:
return body
trimmed = section_text.rstrip("\n")
new_section = trimmed + "\n" + bullet + "\n\n"
return body[:start] + new_section + body[end:]
def add_relationship_bullet(body: str, label: str, other_title: str) -> str: def apply_links_block(page: Page, body: str | None = None) -> str:
bullet = f"- **{label}:** [[{other_title}]]" """`body` with the links region re-rendered from `page.frontmatter`."""
return add_bullet_to_section(body, sections.RELATIONSHIPS, bullet, other_title) return blocks.replace(
page.body if body is None else body, blocks.LINKS, render_links_block(page)
)
def add_see_also_bullet(body: str, other_title: str) -> str: def _check_authorised(source: Page, target: Page, label: str) -> None:
return add_bullet_to_section(body, sections.SEE_ALSO, f"- [[{other_title}]]", other_title) """Refuse a label the source collection has not authorised for that
destination.
Checked here rather than only in `lint` because this is the moment the
author is present: a refusal names the authorised set and can be answered by
picking a better label, while a lint finding a day later is answered by
whoever is holding the report.
"""
source_collection = _collection_of(source)
destination = _collection_of(target)
if source_collection is None or destination is None:
return
allowed = kb_collections.authorised_labels(source_collection, destination)
if not allowed:
fail(
f"kb/{source_collection}/COLLECTION.md authorises no labels for edges into "
f"kb/{destination}/. Add an `outbound:` entry for it, or do not link there "
f"from this collection."
)
if label not in allowed:
fail(
f"'{label}' is not authorised for kb/{source_collection}/ -> kb/{destination}/.\n"
f" Authorised: {', '.join(sorted(allowed))}\n"
f" The catalogue and what each label asserts: instructions/link-taxonomy.md\n"
f" Authorising a further label is a deliberate edit to "
f"kb/{source_collection}/COLLECTION.md, not a way around this refusal."
)
@app.command("add") @app.command("add")
def xref_add( def xref_add(
a: str = typer.Option(..., "--a", help="Exact title of page A"), a: str = typer.Option(..., "--a", help="Exact title of the page that asserts the edge"),
b: str = typer.Option(..., "--b", help="Exact title of page B"), b: str = typer.Option(..., "--b", help="Exact title of the page it points at"),
rel_a: str = typer.Option("related to", "--rel-a", help="Relationship label on A pointing to B"), rel: str = typer.Option(
rel_b: str = typer.Option("related to", "--rel-b", help="Relationship label on B pointing to A"), ..., "--rel", help="Label from instructions/link-taxonomy.md, e.g. depends-on"
see_also: bool = typer.Option(True, "--see-also/--no-see-also", help="Also add reciprocal 'See Also' bullets"), ),
dry_run: bool = typer.Option(False, "--dry-run", help="Preview changes to both pages instead of writing"), dry_run: bool = typer.Option(False, "--dry-run", help="Preview the change instead of writing"),
): ):
"""Declare that A <rel> B. One edge, on A only.
Say the sentence before choosing the label: `[A] <rel> [B]`. If it only
reads true backwards, the edge belongs on B - run this the other way round
rather than reaching for an inverse label.
B is not modified and does not need to point back. Its inbound view is
rendered from the graph.
"""
pages = load_kb_pages(config.KB_DIR) pages = load_kb_pages(config.KB_DIR)
page_a = _find_page(pages, a) page_a = _find_page(pages, a)
page_b = _find_page(pages, b) page_b = _find_page(pages, b)
# Both refusals before either write, so a rejected pair leaves no half-link. # Every refusal before the single write, so a rejected edge leaves nothing.
_require_related_field(page_a, a) _require_related_field(page_a, a)
_require_related_field(page_b, b) _check_authorised(page_a, page_b, rel)
related_changed_a = add_related(page_a.frontmatter, b) changed = links.upsert(page_a.frontmatter, "related", links.Edge(b, rel))
related_changed_b = add_related(page_b.frontmatter, a) body = apply_links_block(page_a)
changed = changed or body != page_a.body
body_a = add_relationship_bullet(page_a.body, rel_a, b)
body_b = add_relationship_bullet(page_b.body, rel_b, a)
if see_also:
body_a = add_see_also_bullet(body_a, b)
body_b = add_see_also_bullet(body_b, a)
changed_a = related_changed_a or body_a != page_a.body
changed_b = related_changed_b or body_b != page_b.body
if dry_run: if dry_run:
state_a = "would update" if changed_a else "already up to date" typer.echo(
state_b = "would update" if changed_b else "already up to date" f"[dry-run] '{a}': {'would declare' if changed else 'already declares'} "
typer.echo(f"[dry-run] '{a}': {state_a} (related / Relationships / See Also)") f"{rel} -> '{b}'"
typer.echo(f"[dry-run] '{b}': {state_b} (related / Relationships / See Also)") )
typer.echo("No files written (--dry-run).") typer.echo("No files written (--dry-run).")
return return
try: if not changed:
write_page(page_a.path, page_a.frontmatter, body_a) success(f"'{a}' already declares {rel} -> '{b}'; nothing changed.")
except OSError as exc: return
fail(f"Failed to write '{a}': {exc}. '{b}' was not touched - fix the write failure and retry once.")
try: try:
write_page(page_b.path, page_b.frontmatter, body_b) write_page(page_a.path, page_a.frontmatter, body)
except OSError as exc: except OSError as exc:
fail( fail(f"Failed to write '{a}': {exc}")
f"'{a}' was updated but writing '{b}' failed: {exc}. The link is now one-directional - " success(f"'{a}' {rel} '{b}'")
f"fix the write failure, then re-run `xref add --a \"{a}\" --b \"{b}\"` (idempotent, safe to retry)."
)
success(f"Linked '{a}' <-> '{b}' ({rel_a} / {rel_b})")
def remove_related(frontmatter: dict, other_title: str) -> bool: def remove_related(frontmatter: dict, other_title: str) -> bool:
"""Drop other_title from frontmatter['related'] if present. Returns True if """Drop every edge pointing at other_title. True if a change was made."""
a change was made.""" return links.remove(frontmatter, "related", other_title)
related = frontmatter.get("related")
if not related or other_title not in related:
return False
frontmatter["related"] = [title for title in related if title != other_title]
return True
def remove_link_bullets(body: str, other_title: str) -> str: def remove_link_bullets(body: str, other_title: str) -> str:
"""Remove the whole-line Relationships/See Also bullets `xref add` writes - """Remove the whole-line Relationships/See Also bullets `xref add` writes -
@@ -223,14 +234,19 @@ def xref_remove(
page_a = _find_page(pages, a) page_a = _find_page(pages, a)
page_b = pages.get(b) page_b = pages.get(b)
body_a = remove_link_bullets(page_a.body, b) # The frontmatter first, then the region re-rendered from it - the body is a
changed_a = strip_frontmatter_ref(page_a, b) or body_a != page_a.body # rendering, so editing the bullet out directly would leave an empty region
# behind and, worse, put the two out of step.
changed_a = strip_frontmatter_ref(page_a, b)
body_a = apply_links_block(page_a, remove_link_bullets(page_a.body, b))
changed_a = changed_a or body_a != page_a.body
changed_b = False changed_b = False
body_b = "" body_b = ""
if page_b is not None: if page_b is not None:
body_b = remove_link_bullets(page_b.body, a) changed_b = strip_frontmatter_ref(page_b, a)
changed_b = strip_frontmatter_ref(page_b, a) or body_b != page_b.body body_b = apply_links_block(page_b, remove_link_bullets(page_b.body, a))
changed_b = changed_b or body_b != page_b.body
if dry_run: if dry_run:
typer.echo(f"[dry-run] '{a}': {'would update' if changed_a else 'no reference to remove'}") typer.echo(f"[dry-run] '{a}': {'would update' if changed_a else 'no reference to remove'}")
@@ -278,7 +294,11 @@ def xref_link_source(
sources = page.frontmatter.setdefault("sources", []) sources = page.frontmatter.setdefault("sources", [])
if source not in sources: if source not in sources:
sources.append(source) sources.append(source)
body = add_see_also_bullet(page.body, source) # No body bullet. `sources:` *is* the record, and the See Also bullet
# this used to add was the reciprocal half of a bidirectional model
# that no longer exists - 353 of the corpus's 555 such bullets were
# provably redundant with an edge that already said the same thing.
body = page.body
# The way back. Until this existed the command wrote only the targets, # The way back. Until this existed the command wrote only the targets,
# so a source page's own `entities:`/`concepts:` stayed as `new` left # so a source page's own `entities:`/`concepts:` stayed as `new` left
+164 -16
View File
@@ -4,32 +4,164 @@ The repo is a pipeline: `raw/` (untrusted input) -> `types/` + `tools/` (schema
compiler) -> `kb/` (compiled knowledge) -> `reports/` (derived output). Only `kb/` is compiler) -> `kb/` (compiled knowledge) -> `reports/` (derived output). Only `kb/` is
divided into collections; the other three stages are single-purpose directories. divided into collections; the other three stages are single-purpose directories.
Repo root is resolved by walking up from this file's location (tools/chemenu/config.py Repo root is resolved in three steps - an explicit argument to `resolve_root()`, then
-> tools/ -> repo root), which is stable regardless of the caller's current working `$CHEMENU_ROOT`, then a walk up from this file's location (tools/chemenu/config.py ->
directory. tools/ -> repo root). The walk-up stays the default, so `tools/wikitool` behaves exactly
as it always has; the two steps in front of it are what lets an in-process caller point
this package at a corpus it does not itself live inside.
**Nothing below is bound at import time.** `ROOT` and every path derived from it are
resolved on each attribute access, through the module `__getattr__` at the bottom. They
used to be module constants, which had a failure mode worse than the limitation itself:
`monkeypatch.setattr(config, "ROOT", other)` repointed `ROOT` and left `KB_DIR` and
`RAW_DIR` aimed at wherever this file happens to sit, so a caller that believed it was
working on a target tree was in fact answering out of the developer's checkout. Resolving
on access makes the derived paths follow whatever `ROOT` currently is - including a
monkeypatched one - so the half-repointed state cannot be constructed.
""" """
import os import os
import subprocess import subprocess
from contextlib import contextmanager
from pathlib import Path from pathlib import Path
ROOT = Path(__file__).resolve().parents[2] # The last resort, and the default every existing caller gets: the checkout this
# file is part of.
_PACKAGE_ROOT = Path(__file__).resolve().parents[2]
RAW_DIR = ROOT / "raw" # Points this package at a corpus other than its own checkout. Registered in
KB_DIR = ROOT / "kb" # `_WIKITOOL_ENV` (tools/chemenu/tests/conftest.py), so the suite runs with it
TYPES_DIR = ROOT / "types" # cleared and a test that wants it sets it itself.
REPORTS_DIR = ROOT / "reports" ENV_ROOT = "CHEMENU_ROOT"
WORK_DIR = ROOT / "work"
INSTRUCTIONS_DIR = ROOT / "instructions"
def resolve_root(explicit: "Path | str | None" = None) -> Path:
"""The repo root, by the documented precedence: argument, then
`$CHEMENU_ROOT`, then the checkout this package lives in.
An explicit argument wins because a caller serving two corpora cannot use a
process-wide variable to tell them apart; the variable exists for the case
where the caller is a whole process (a server, a CI job) and there is
nothing to pass it through.
"""
if explicit is not None:
return Path(explicit).expanduser().resolve()
from_env = os.environ.get(ENV_ROOT, "").strip()
if from_env:
return Path(from_env).expanduser().resolve()
return _PACKAGE_ROOT
def _root() -> Path:
"""`ROOT` as it stands right now, honouring an assignment onto this module.
Reads the module dict directly rather than `resolve_root()` so that a test
(or any caller) setting `config.ROOT` is what the derived paths follow.
That assignment is why the derived paths are computed here at all.
"""
assigned = globals().get("ROOT")
return Path(assigned) if assigned is not None else resolve_root()
# Everything under the root, as a name -> relative-path table rather than as
# assignments. One place to read, and the only place that has to know a derived
# path exists at all.
_DERIVED = {
"RAW_DIR": ("raw",),
"KB_DIR": ("kb",),
"TYPES_DIR": ("types",),
"REPORTS_DIR": ("reports",),
"WORK_DIR": ("work",),
"INSTRUCTIONS_DIR": ("instructions",),
# Generated copies of the skill directories under `instructions/`. Both are # Generated copies of the skill directories under `instructions/`. Both are
# gitignored: they are build output, and a fresh clone publishes them with # gitignored: they are build output, and a fresh clone publishes them with
# `wikitool instructions sync` (see instructions/bootstrap.md). # `wikitool instructions sync` (see instructions/bootstrap.md).
AGENTS_SKILLS_DIR = ROOT / ".agents" / "skills" "AGENTS_SKILLS_DIR": (".agents", "skills"),
CLAUDE_SKILLS_DIR = ROOT / ".claude" / "skills" "CLAUDE_SKILLS_DIR": (".claude", "skills"),
}
INDEX_FILE = KB_DIR / "index.md" # The generated files, derived from `KB_DIR` rather than from the root: a
LOG_FILE = KB_DIR / "log.md" # caller that repoints only the corpus directory must not be left with a log
PROVENANCE_FILE = KB_DIR / "provenance.md" # and a catalog belonging to a different tree.
_KB_DERIVED = {
"INDEX_FILE": "index.md",
"LOG_FILE": "log.md",
"PROVENANCE_FILE": "provenance.md",
}
# Every name this module resolves rather than stores. Assigning one is
# supported - that is what makes the paths repointable at all - but the
# assignment has to be taken back afterwards, or it outlives the caller that
# made it. See `reset()`.
MANAGED_PATHS = ("ROOT", *_DERIVED, *_KB_DERIVED)
@contextmanager
def rooted(root: "Path | str"):
"""Resolve every managed path under `root` for the duration of the block.
Some things below the read core reach for `config` directly rather than
taking a root - the module-level `TypeResolver` singleton, which has to
find `types/`, is the one that matters - so pointing this package at
another corpus means pointing `config` at it, not only the functions that
accept an argument.
**Process-wide while it is open, and therefore not thread-safe.** A caller
serving several corpora at once holds a lock around it, the same discipline
`CorpusCache` documents. That is a real constraint and not a hidden one:
`$CHEMENU_ROOT` is process-wide for the same reason, and the server this
exists for (Gitea #19) serves one checkout that a `git reset --hard` keeps
clean.
Restores exactly what was there, including "nothing was assigned" - it must
not leave `ROOT` bound behind it, or it recreates the stale-binding bug in
the shape `reset()` describes.
"""
previous = {name: globals()[name] for name in MANAGED_PATHS if name in globals()}
reset()
globals()["ROOT"] = Path(root)
try:
yield Path(root)
finally:
reset()
globals().update(previous)
def reset() -> None:
"""Drop every assignment onto a managed path name, back to resolution.
The test suite calls this between tests, and it is not optional there.
`monkeypatch.setattr(config, "KB_DIR", tmp)` records the old value by
*reading* it - which resolves it - and its undo then writes that resolved
path back as a real attribute. The name is bound from then on, so the next
caller to repoint only `ROOT` gets a `KB_DIR` still aimed at the previous
tree: exactly the half-repointed state this module was rewritten to make
unconstructible, rebuilt by the cleanup rather than by the test.
"""
for name in MANAGED_PATHS:
globals().pop(name, None)
def __getattr__(name: str):
"""Resolve `ROOT` and the paths under it on access (PEP 562).
Only reached for names *not* in the module dict, so an explicit assignment
- `monkeypatch.setattr(config, "ROOT", tmp)` - keeps working and now also
carries the derived paths with it, which is the bug this replaces.
"""
if name == "ROOT":
return resolve_root()
if name in _DERIVED:
return _root().joinpath(*_DERIVED[name])
if name in _KB_DERIVED:
kb_dir = globals().get("KB_DIR")
base = Path(kb_dir) if kb_dir is not None else _root() / "kb"
return base / _KB_DERIVED[name]
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
def __dir__() -> list[str]:
return sorted([*globals(), "ROOT", *_DERIVED, *_KB_DERIVED])
# Files/patterns to ignore when scanning raw/ for ingest coverage. # Files/patterns to ignore when scanning raw/ for ingest coverage.
# CONTRACT.md is the layer's source contract, not source material. # CONTRACT.md is the layer's source contract, not source material.
@@ -70,6 +202,22 @@ ENVIRONMENT_TEMPLATE = f"{ENVIRONMENT_FILE}.template"
# `dist export` already computes (AGENTS.md invariant 8). See NOTICE. # `dist export` already computes (AGENTS.md invariant 8). See NOTICE.
LICENSE_FILES = ("LICENSE", "LICENSE-CONTENT", "NOTICE") LICENSE_FILES = ("LICENSE", "LICENSE-CONTENT", "NOTICE")
# Which push targets `publish` may write to, for a checkout that says so. The
# danger this addresses is one checkout's content reaching another checkout's
# remote - a private instance pushing its own `kb/` to a public upstream, where
# it cannot be taken back.
#
# It pins **URLs, not remote names**: a name-based list would pass a `publish`
# whose `origin` had been repointed, which is the failure it exists to catch.
#
# Per-checkout and gitignored, like `ENVIRONMENT.md` and for the same reason:
# two clones of this repo push to two different places, so a committed copy
# would hand the second one an answer that is wrong rather than missing. Absent
# means unrestricted - `doctor` reports it, and the Publish-Remote Gate simply
# does not apply. A checkout that holds private content should have one; see
# instructions/gates.md.
PUBLISH_REMOTES_FILENAME = ".wikitool-remotes.json"
def default_author() -> str | None: def default_author() -> str | None:
"""The author to stamp a new source page with, per instance. """The author to stamp a new source page with, per instance.
@@ -86,7 +234,7 @@ def default_author() -> str | None:
try: try:
result = subprocess.run( result = subprocess.run(
["git", "config", "user.name"], ["git", "config", "user.name"],
cwd=ROOT, cwd=_root(),
capture_output=True, capture_output=True,
text=True, text=True,
timeout=5, timeout=5,
+177
View File
@@ -0,0 +1,177 @@
"""What this instance decided, read from `kb/CONVENTIONS.md`.
`kb/CONTRACT.md` and this file answer two different questions. The contract
holds what the code enforces - what a collection is, which files are generated,
how `provenance:` and `confidence_base` work - and is identical in every
instance, so `dist export` ships it verbatim. `kb/CONVENTIONS.md` holds what
each instance decides for itself: the language its pages are written in, the
relationship-label vocabulary, the tone examples, the confidence rubric, the
ADR prefix. The distribution ships only `kb/CONVENTIONS.md.template`, exactly
the split `USER.md`/`SOUL.md` already use one directory up.
Only one part of it is machine-read, and it is the part that used to be Python:
the three section headings `xref add` and `cite add` write. While
`RELATIONSHIPS = "Beziehungen"` sat in `sections.py`, an instance writing its
pages in any other language had to edit the compiler to say so - which made the
KB language a stack property in code while every document called it an instance
decision.
**A missing conventions file is not an error here.** It is the state an
instance is in between installing this machinery and running the migration that
writes the file, and every command has to keep working through it. The fallback
is `PRE_CONVENTIONS_NAMES` - not "the stack's language", but *what this stack
hardcoded before the file existed*, which is by construction what any corpus
reaching that state was written with. `wikitool doctor` is what says the file is
missing; degrading loudly here would take out `doctor` itself.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Optional
from chemenu import config
from chemenu.frontmatter_io import read_page
CONVENTIONS_FILENAME = "CONVENTIONS.md"
CONVENTIONS_TEMPLATE = f"{CONVENTIONS_FILENAME}.template"
# The two tool-owned regions, keyed by the block name in `chemenu.blocks`. The
# block name is the identifier - it is what the marker pair carries and what the
# tool locates the region by - while the heading text below it is prose the
# instance chooses.
#
# `see_also` is gone as a section: it was the reciprocal half of the old
# bidirectional `xref add`, and under authored directional edges it is a *label*
# inside the links block rather than a region of its own.
SECTIONS_KEY = "sections"
LANGUAGE_KEY = "language"
# What a heading renders as when the instance has not said. Purely cosmetic, and
# that is a genuine change from before: while the tool located a region by
# matching this text, a wrong default silently split a page into two sections and
# `xref add` appended to the wrong one. Now the marker pair carries the identity,
# so a region rendered under the wrong words is a *display* fault that the next
# write repairs by itself once `kb/CONVENTIONS.md` says otherwise.
#
# So this is a fallback for the window between installing the machinery and
# writing the conventions file - `doctor` is what makes that window loud - and
# not a language the compiler has an opinion about.
DEFAULT_HEADINGS: dict[str, str] = {
"links": "Relationships",
"footnotes": "Footnotes",
}
def conventions_file() -> Path:
return config.KB_DIR / CONVENTIONS_FILENAME
# (path, mtime_ns, size) -> frontmatter. `heading_re()` is called once per page
# per lint run, so re-reading the file each time would put a stat+parse on a
# per-page path for a document that changes about once per instance. Keyed on
# the stat rather than on the path alone, so a test that rewrites the file
# inside one process is not answered out of the cache.
_CACHE: dict[tuple[str, int, int], dict[str, Any]] = {}
def read_conventions() -> dict[str, Any]:
"""`kb/CONVENTIONS.md`'s frontmatter, or `{}` if the file is absent.
Permissive on purpose, like `read_page` itself: a conventions file with
broken YAML degrades to the pre-conventions defaults rather than taking
every command down with it. `doctor` and `docs verify` are where that
surfaces as a finding.
"""
path = conventions_file()
if not path.is_file():
return {}
stat = path.stat()
key = (str(path), stat.st_mtime_ns, stat.st_size)
if key not in _CACHE:
frontmatter, _ = read_page(path)
_CACHE.clear()
_CACHE[key] = frontmatter
return _CACHE[key]
def reset_cache() -> None:
"""Drop the parsed conventions. For a caller that rewrote the file and has
to see the new value within the same stat resolution."""
_CACHE.clear()
def _mapping(key: str) -> dict[str, Any]:
value = read_conventions().get(key)
return value if isinstance(value, dict) else {}
def language() -> Optional[str]:
"""The declared KB language tag (e.g. `de`), or None if undeclared.
Nothing in the compiler branches on it - the language is carried by the
prose the instance writes, not by a switch. It is here because the
conventions file is where a human and an agent look the answer up, and
because `doctor` reports it.
"""
value = read_conventions().get(LANGUAGE_KEY)
if value is None:
return None
return str(value).strip() or None
def heading(block: str) -> str:
"""The heading this instance renders above `block`'s generated region."""
declared = _mapping(SECTIONS_KEY).get(block)
if isinstance(declared, str) and declared.strip():
return declared.strip()
return DEFAULT_HEADINGS.get(block, block.title())
def declaration_issues() -> list[str]:
"""What is wrong with this instance's conventions file, if anything.
Shared by `doctor` (which FAILs on it) and `docs verify` (which refuses a
tree with it), so the two cannot disagree about what a valid declaration
looks like. An absent file is *not* reported here - that is a separate
finding with a separate fix, and only `doctor` makes it one.
"""
from chemenu import blocks
path = conventions_file()
if not path.is_file():
return []
issues: list[str] = []
frontmatter, _ = read_page(path)
if not frontmatter:
return [
f"kb/{CONVENTIONS_FILENAME} has no readable frontmatter - it must declare "
f"`{SECTIONS_KEY}:` with the headings this instance renders"
]
declared = frontmatter.get(SECTIONS_KEY)
if not isinstance(declared, dict):
return [
f"kb/{CONVENTIONS_FILENAME}: `{SECTIONS_KEY}:` must be a mapping of "
f"{'/'.join(blocks.BLOCKS)} to the heading this instance renders above it"
]
for block in blocks.BLOCKS:
value = declared.get(block)
if not isinstance(value, str) or not value.strip():
issues.append(
f"kb/{CONVENTIONS_FILENAME}: `{SECTIONS_KEY}.{block}` is missing or empty - "
f"the generated `{block}` region would render under a default heading rather "
"than this instance's own"
)
for block in sorted(set(declared) - set(blocks.BLOCKS)):
issues.append(
f"kb/{CONVENTIONS_FILENAME}: `{SECTIONS_KEY}.{block}` is not a region the tool "
f"generates; the regions are {', '.join(blocks.BLOCKS)}"
)
if config.TEMPLATE_SENTINEL in path.read_text(encoding="utf-8"):
issues.append(
f"kb/{CONVENTIONS_FILENAME} still carries the `{config.TEMPLATE_SENTINEL}` line - "
"a renamed template is not a filled one"
)
return issues
+123
View File
@@ -0,0 +1,123 @@
"""Load the corpus once per revision instead of once per query.
`load_pages_by_path()` reads and parses every page under `kb/` on every call.
For a CLI that is the right shape - one call per process, nothing to reuse, and
a cache would only add a way to answer from a tree that has since changed. For
a long-lived reader (the MCP server) it is the opposite: the same corpus is
reparsed for every request, and the cost grows linearly with the corpus.
So the cache is an object a caller holds, not a module-level dict that switches
itself on behind everyone's back. The CLI holds none and behaves exactly as
before; a resident process holds one.
**The key is the commit SHA, and a dirty tree is never cached.** The SHA alone
would be a correctness bug in any checkout someone edits: a session that writes
a page and searches for it would be answered from the parse taken before the
write, with nothing about the SHA having changed. A clean tree is the state the
server actually runs in - it is kept that way by `git fetch && git reset
--hard` - so the fast path is the one that holds there, and every other tree
falls back to reloading.
That same SHA is what a response is stamped with, which is deliberate: the
revision a caller is told about is by construction the revision its answer was
computed from.
"""
from __future__ import annotations
import subprocess
from pathlib import Path
from typing import Optional
from chemenu import config
from chemenu.page import Page
def head_commit(root: Optional[Path] = None) -> Optional[str]:
"""The full SHA of `HEAD`, or None outside a git checkout."""
result = _git(["rev-parse", "HEAD"], root)
if result is None or result.returncode != 0:
return None
return result.stdout.strip() or None
def is_dirty(root: Optional[Path] = None, path: Optional[Path] = None) -> bool:
"""Whether the working tree has uncommitted changes under `path`.
Errs toward dirty: if git cannot answer, the answer is "assume it changed".
A cache that treats "unknown" as clean serves stale pages, which is the one
outcome this module exists to prevent.
"""
root = root or config.ROOT
target = path or config.KB_DIR
try:
relative = Path(target).resolve().relative_to(Path(root).resolve()).as_posix()
except ValueError:
return True
result = _git(["status", "--porcelain", "--", relative], root)
if result is None or result.returncode != 0:
return True
return bool(result.stdout.strip())
def _git(args: list[str], root: Optional[Path] = None):
try:
return subprocess.run(
["git", *args],
cwd=root or config.ROOT,
capture_output=True,
text=True,
timeout=10,
check=False,
)
except (OSError, subprocess.SubprocessError):
return None
class CorpusCache:
"""One parsed corpus, reused while the checkout stays on the same commit.
Not thread-safe by itself: a caller serving concurrent requests holds the
lock. Kept out of here because the locking discipline belongs to whoever
owns the request loop, and a lock hidden in a cache is one nobody can see
when they need to reason about it.
"""
def __init__(self, kb_dir: Optional[Path] = None, root: Optional[Path] = None):
self.kb_dir = kb_dir
self.root = root
self._pages: Optional[dict[str, Page]] = None
self._revision: Optional[str] = None
@property
def revision(self) -> Optional[str]:
"""The commit the cached corpus was read at, or None if nothing is
cached (including because the tree was dirty)."""
return self._revision
def current_revision(self) -> Optional[str]:
"""The commit this corpus would be cached under right now: `HEAD` on a
clean tree, None on a dirty one or outside git. None means uncacheable,
which is why it is also what a caller should report as "no revision" -
an answer read out of a dirty tree does not correspond to any commit."""
root = self.root or config.ROOT
if is_dirty(root, self.kb_dir or config.KB_DIR):
return None
return head_commit(root)
def load(self) -> tuple[dict[str, Page], Optional[str]]:
"""(pages, revision). Reparses whenever the revision is not the cached
one, and on every call while the tree is dirty."""
# Imported here rather than at module level: `commands.search` pulls in
# typer, and this module is meant to be importable by a library caller
# that has no CLI. Removing that edge properly is the library-boundary
# work, not this file's job.
from chemenu.commands.search import load_pages_by_path
revision = self.current_revision()
if revision is None or revision != self._revision or self._pages is None:
pages = load_pages_by_path(self.kb_dir, self.root)
if revision is None:
self._pages, self._revision = None, None
return pages, None
self._pages, self._revision = pages, revision
return self._pages, self._revision
+20 -1
View File
@@ -26,7 +26,7 @@ from collections import Counter
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any, Optional from typing import Any, Optional
from chemenu import kb_scan, provenance from chemenu import blocks, kb_scan, provenance
from chemenu.page import Page from chemenu.page import Page
from chemenu.type_resolver import resolver from chemenu.type_resolver import resolver
@@ -59,6 +59,7 @@ class PageShape:
cite_refs: Counter cite_refs: Counter
cite_defs: dict[str, str] cite_defs: dict[str, str]
fields: dict[str, Any] fields: dict[str, Any]
markers: dict[str, int]
body: str body: str
@classmethod @classmethod
@@ -81,6 +82,7 @@ class PageShape:
cite_refs=Counter(m.group(1) for m in provenance.CITE_REF_RE.finditer(head)), cite_refs=Counter(m.group(1) for m in provenance.CITE_REF_RE.finditer(head)),
cite_defs={cite_id: source for cite_id, (source, _) in definitions.items()}, cite_defs={cite_id: source for cite_id, (source, _) in definitions.items()},
fields=fields, fields=fields,
markers=blocks.marker_pairs(page.body),
body=page.body, body=page.body,
) )
@@ -163,6 +165,23 @@ def compare_page(path: str, before: PageShape, after: PageShape) -> list[PageFin
changed.append(f"[^{cite_id}] {was!r} -> {now!r}") changed.append(f"[^{cite_id}] {was!r} -> {now!r}")
findings.append(PageFinding(path, "cite-defs", ", ".join(changed))) findings.append(PageFinding(path, "cite-defs", ", ".join(changed)))
# A generated region that lost or gained a marker is the failure mode the
# delimiters were introduced against, and it is silent: a lost opening
# marker turns the region into ordinary prose, and the next write appends a
# second region beside it. An agent rewriting prose at the boundary is
# exactly how that happens, which is what makes it a migration invariant
# rather than a lint nicety.
#
# Counts, not presence - the same reasoning as the wikilink counter. A page
# that goes from one links region to two has the same *set* of region names.
if before.markers != after.markers:
changed_regions = []
for name in sorted(set(before.markers) | set(after.markers)):
was, now = before.markers.get(name, 0), after.markers.get(name, 0)
if was != now:
changed_regions.append(f"{name}: {was} -> {now}")
findings.append(PageFinding(path, "markers", ", ".join(changed_regions)))
changed_fields = [] changed_fields = []
for name in sorted(set(before.fields) | set(after.fields)): for name in sorted(set(before.fields) | set(after.fields)):
was, now = before.fields.get(name), after.fields.get(name) was, now = before.fields.get(name), after.fields.get(name)
+31
View File
@@ -0,0 +1,31 @@
"""The exception contract at the library boundary.
The CLI reports a bad argument by printing an `ERROR` line and leaving through
`typer.Exit(1)`. That is the right answer for a terminal and the wrong one for
an in-process caller, which gets an exit code where it expected a value, plus
module-global state (`_util._declined`) surviving into its next call.
So the core raises, and the CLI adapter translates. `ChemenuError` is the one
class a library caller has to know; the two below it separate "your input was
wrong, a different argument would work" from "the machinery underneath failed",
which is the same distinction the CLI's exit codes draw.
`ValidationError` also inherits `ValueError`. Not for elegance: `PredicateError`
was a `ValueError` before this existed, and callers catch it that way.
"""
from __future__ import annotations
class ChemenuError(Exception):
"""Base for every error this package raises deliberately."""
class ValidationError(ChemenuError, ValueError):
"""The caller's input was rejected. Re-running unchanged fails identically;
the CLI renders this as its exit-1 `ERROR` line."""
class BackendError(ChemenuError, RuntimeError):
"""A dependency the core relies on was missing or failed - `rg` absent, a
search that had to be killed. Not the caller's argument, and not
necessarily permanent."""
+156 -30
View File
@@ -16,29 +16,137 @@ from typing import Any
import yaml import yaml
from chemenu.errors import ValidationError
try: # pragma: no cover - which branch runs depends on the host's libyaml
from yaml import CSafeLoader as _Loader
except ImportError: # pragma: no cover
from yaml import SafeLoader as _Loader
FRONTMATTER_RE = re.compile(r"\A---\n(.*?)\n---\n?(.*)\Z", re.DOTALL) FRONTMATTER_RE = re.compile(r"\A---\n(.*?)\n---\n?(.*)\Z", re.DOTALL)
# Frontmatter is a flat block of scalars and one-line lists. Real pages sit
# well under a kilobyte, so this is not a budget anyone writes against - it is
# the bound that keeps parse cost proportional to the corpus rather than to
# whatever a single file claims to be. It matters because the parser is on the
# read path, and the read path is what gets exposed.
MAX_FRONTMATTER_BYTES = 64 * 1024
# YAML anchors and aliases are refused outright rather than budgeted. A page's
# frontmatter has no use for them, and alias expansion is where a small
# document becomes an enormous object graph: 267 bytes of nested aliases
# compose in 0.2 ms into 672,603 nodes, growing 9**n with nesting depth at
# constant parse time. A size limit alone does not touch that, because the
# input stays small - see `ALIAS_BOMB` in the tests.
#
# The check runs on the *event* stream (`yaml.parse`), which is streaming and
# resolves nothing - so asking the question costs O(text) and never triggers
# the expansion it is asking about. `"*"` is a necessary character in any alias
# node, so its absence proves absence without parsing at all, which is the case
# every real page takes.
_ALIAS_HINT = "*"
class FrontmatterError(ValidationError):
"""Frontmatter that cannot be used: missing, malformed, oversized, or
refused by a limit. Raised only by the strict entry points - the permissive
ones report it as a string instead."""
def _load_frontmatter(fm_text: str) -> tuple[dict[str, Any] | None, str | None]:
"""Parse one frontmatter block into (mapping, error). Exactly one is None.
The single parser behind both `read_page()` and `frontmatter_error()`. They
used to have one each, which is how the permissive path could degrade to
`{}` for a reason the strict path described differently - and how every
caller wanting both answers read and parsed the file twice.
"""
encoded = len(fm_text.encode("utf-8"))
if encoded > MAX_FRONTMATTER_BYTES:
return None, (
f"frontmatter is {encoded} bytes, over the {MAX_FRONTMATTER_BYTES}-byte limit"
)
if _ALIAS_HINT in fm_text:
try:
for event in yaml.parse(fm_text, Loader=_Loader):
if isinstance(event, yaml.AliasEvent):
return None, (
"frontmatter uses a YAML alias (`*"
f"{event.anchor}`); anchors and aliases are not allowed here"
)
except yaml.YAMLError as exc:
return None, f"invalid YAML frontmatter: {_first_line(exc)}"
except RecursionError:
return None, "frontmatter is nested too deeply to parse"
try:
parsed = yaml.load(fm_text, Loader=_Loader)
except yaml.YAMLError as exc:
return None, f"invalid YAML frontmatter: {_first_line(exc)}"
except RecursionError:
# PyYAML composes recursively, so deep nesting exhausts the interpreter
# stack rather than raising a YAMLError. Unbounded nesting is bounded by
# MAX_FRONTMATTER_BYTES; this catches what fits under it.
return None, "frontmatter is nested too deeply to parse"
if parsed is None:
return None, "empty frontmatter block"
if not isinstance(parsed, dict):
return None, f"frontmatter is {type(parsed).__name__}, expected a mapping"
return parsed, None
def _first_line(exc: Exception) -> str:
text = str(exc)
return text.splitlines()[0] if text else exc.__class__.__name__
def read_page_with_error(path: Path) -> tuple[dict[str, Any], str, str | None]:
"""(frontmatter, body, error) - the permissive read, with the reason it was
permissive handed back instead of dropped.
A page whose YAML is broken reads as `{}`, and a `{}` page then has no
`confidence` and no `kind`: it drops out of `--field confidence<0.6` -
precisely the query meant to find pages in bad shape - while looking to the
caller like a page that simply did not match. Returning the reason is what
lets a caller say so instead of losing the page quietly.
"""
text = path.read_text(encoding="utf-8")
match = FRONTMATTER_RE.match(text)
if not match:
return {}, text, "no `---` frontmatter block"
fm_text, body = match.group(1), match.group(2)
parsed, error = _load_frontmatter(fm_text)
if error == "empty frontmatter block":
# An empty block is a legitimate shape for the permissive read - it
# carries no fields, and there is nothing to lose. Only `lint` treats
# it as a finding.
return {}, body, error
return (parsed or {}), body, error
def read_page(path: Path) -> tuple[dict[str, Any], str]: def read_page(path: Path) -> tuple[dict[str, Any], str]:
"""Return (frontmatter_dict, body) for a markdown file. If the file has no """Return (frontmatter_dict, body) for a markdown file. If the file has no
frontmatter block, returns ({}, full_text). frontmatter block, returns ({}, full_text).
Deliberately permissive: malformed YAML degrades to an empty dict so bulk Deliberately permissive: malformed YAML degrades to an empty dict so bulk
operations never crash on one bad page. Use `frontmatter_error()` (which operations never crash on one bad page. Use `read_page_with_error()` (search
`wikitool lint` does) to surface those pages instead of losing them does) or `frontmatter_error()` (`wikitool lint` does) to surface those pages
silently. instead of losing them silently.
""" """
text = path.read_text(encoding="utf-8") frontmatter, body, _ = read_page_with_error(path)
match = FRONTMATTER_RE.match(text) return frontmatter, body
if not match:
return {}, text
fm_text, body = match.group(1), match.group(2) def read_page_strict(path: Path) -> tuple[dict[str, Any], str]:
try: """`read_page()` that raises `FrontmatterError` instead of degrading.
frontmatter = yaml.safe_load(fm_text) or {}
except yaml.YAMLError: For any path that ingests frontmatter this instance did not write itself.
frontmatter = {} The permissive read is right for bulk operations over a corpus the operator
if not isinstance(frontmatter, dict): committed; it is wrong the moment the document arrives from outside, where
frontmatter = {} "unparseable" must stop the document rather than empty it.
"""
frontmatter, body, error = read_page_with_error(path)
if error is not None:
raise FrontmatterError(f"{path}: {error}")
return frontmatter, body return frontmatter, body
@@ -50,20 +158,7 @@ def frontmatter_error(path: Path) -> str | None:
YAML is malformed (or whose frontmatter block is missing entirely) reads YAML is malformed (or whose frontmatter block is missing entirely) reads
back as `{}` and then quietly slips past every frontmatter-driven check. back as `{}` and then quietly slips past every frontmatter-driven check.
""" """
text = path.read_text(encoding="utf-8") return read_page_with_error(path)[2]
match = FRONTMATTER_RE.match(text)
if not match:
return "no `---` frontmatter block"
try:
parsed = yaml.safe_load(match.group(1))
except yaml.YAMLError as exc:
reason = str(exc).splitlines()[0] if str(exc) else exc.__class__.__name__
return f"invalid YAML frontmatter: {reason}"
if parsed is None:
return "empty frontmatter block"
if not isinstance(parsed, dict):
return f"frontmatter is {type(parsed).__name__}, expected a mapping"
return None
def _format_scalar(value: Any, flow: bool = False) -> str: def _format_scalar(value: Any, flow: bool = False) -> str:
@@ -132,7 +227,7 @@ def _round_trips_as_string(text: str, flow: bool = False) -> bool:
""" """
probe, expected = (f"[{text}]", [text]) if flow else (text, text) probe, expected = (f"[{text}]", [text]) if flow else (text, text)
try: try:
return yaml.safe_load(probe) == expected return yaml.load(probe, Loader=_Loader) == expected
except yaml.YAMLError: except yaml.YAMLError:
# Unparseable bare - quoting is exactly the fix. # Unparseable bare - quoting is exactly the fix.
return False return False
@@ -172,10 +267,41 @@ def _format_list(items: list[Any]) -> str:
return "[" + ", ".join(_format_scalar(v, flow=True) for v in items) + "]" return "[" + ", ".join(_format_scalar(v, flow=True) for v in items) + "]"
def _is_single_key_mapping(value: Any) -> bool:
return isinstance(value, dict) and len(value) == 1
def _format_mapping_list(key: str, items: list[Any]) -> str:
"""A list holding `label: target` pairs, rendered block-style.
The inline `[...]` form this file uses everywhere else cannot carry a
mapping without quoting rules nobody reading the file would guess, so a
labelled edge list is the one place block style earns its keep:
related:
- depends-on: Hermes
- Borealis
Bare strings mixed in stay bare - that is an edge whose label has not been
declared yet, and promoting it to some default here would erase exactly what
`lint` is looking for.
"""
lines = [f"{key}:"]
for item in items:
if _is_single_key_mapping(item):
(label, target), = item.items()
lines.append(f" - {_format_scalar(label)}: {_format_scalar(target)}")
else:
lines.append(f" - {_format_scalar(item)}")
return "\n".join(lines)
def dump_frontmatter(frontmatter: dict[str, Any]) -> str: def dump_frontmatter(frontmatter: dict[str, Any]) -> str:
lines = [] lines = []
for key, value in frontmatter.items(): for key, value in frontmatter.items():
if isinstance(value, list): if isinstance(value, list) and any(_is_single_key_mapping(v) for v in value):
lines.append(_format_mapping_list(key, value))
elif isinstance(value, list):
lines.append(f"{key}: {_format_list(value)}") lines.append(f"{key}: {_format_list(value)}")
else: else:
lines.append(f"{key}: {_format_scalar(value)}") lines.append(f"{key}: {_format_scalar(value)}")
+165
View File
@@ -20,11 +20,77 @@ Two corollaries are enforced rather than documented:
from __future__ import annotations from __future__ import annotations
from pathlib import Path from pathlib import Path
from typing import Any
from chemenu import config from chemenu import config
CONTRACT_NAME = "COLLECTION.md" CONTRACT_NAME = "COLLECTION.md"
# What a collection declares about itself, in `COLLECTION.md`'s frontmatter.
#
# Presence on the filesystem says a collection *exists*; it cannot say who owns
# the rules inside it. A `COLLECTION.md` is instance-owned - the distribution
# ships a `.template` per default collection and the instance writes the real
# one - so the two facts the stack still needs from it have to be declared
# rather than inferred from the directory name, which an instance is free to
# choose.
PROFILE_FIELD = "profile"
REQUIRED_BY_STACK_FIELD = "required_by_stack"
# Which link labels a page in this collection may use, per destination
# collection. The **source** collection decides, which is the whole point: an
# edge is an authored reader-aid written on the page that asserts it, so the
# rules that govern it are the rules of the collection that page lives in. A
# destination is another collection's name, or `any`.
#
# This is Commonplace's ADR-019 adopted directly, and it is what makes a
# 35-label catalogue usable: a collection authorises the six that make sense
# from it, and the rest of the palette is simply not on its menu.
OUTBOUND_FIELD = "outbound"
ANY_DESTINATION = "any"
# The types `wikitool` itself depends on existing, as opposed to ones an
# instance keeps because they are useful. `source` is here because the whole
# `raw/ -> kb/` provenance path is built on it: `sources coverage` asks which
# raw files no source page claims, every `[^cite-id]` resolves to a source page,
# and `sources rebuild-index` writes `kb/provenance.md` from them. All three ask
# `page.kind == "source"`, so what is load-bearing is the type-spec's `name:`
# and its schema requiring `raw_files:` - not the directory, not the title
# prefix, and not a word of its prose or its template.
#
# That is the whole anchor, and it is deliberately this small: the four page
# type-specs belong to the instance (see types/type-spec.md), so anything more
# would be the stack reaching into a file it does not own.
STACK_REQUIRED_TYPES = ("source",)
STACK_REQUIRED_TYPE_FIELDS = {"source": ("raw_files",)}
def stack_required_collections() -> tuple[str, ...]:
"""Collection names an instance may not rename or drop.
**Derived, not listed.** The required collection is whichever one the
required type writes into - so an instance that legitimately renames
`kb/sources/` to something else, and says so in the type-spec's `base_dir:`,
stays consistent instead of tripping a constant that hardcoded the old name.
A second literal list would only be a copy that drifts.
"""
from chemenu.type_resolver import resolver
names: list[str] = []
for type_name in STACK_REQUIRED_TYPES:
try:
type_path = resolver.find_type_by_name(type_name)
if not type_path:
continue
if resolver.get_root(type_path) != "kb":
continue
base_dir = resolver.get_base_dir(type_path)
except (ValueError, OSError):
continue
if base_dir:
names.append(str(base_dir).strip("/"))
return tuple(dict.fromkeys(names))
def iter_kb_collections(kb_dir: Path | None = None) -> list[Path]: def iter_kb_collections(kb_dir: Path | None = None) -> list[Path]:
"""Return every collection directory under kb/, sorted by name. """Return every collection directory under kb/, sorted by name.
@@ -82,6 +148,105 @@ def stray_collection_contracts(root: Path | None = None, kb_dir: Path | None = N
return sorted(stray) return sorted(stray)
def collection_declaration(collection: Path) -> dict[str, Any]:
"""A collection's own `COLLECTION.md` frontmatter, or `{}` if it has none.
Permissive like every other frontmatter read in this package: an unreadable
declaration degrades to empty here and is reported by `docs verify`, rather
than taking down the discovery every command starts with.
"""
from chemenu.frontmatter_io import read_page
contract = collection / CONTRACT_NAME
if not contract.is_file():
return {}
frontmatter, _ = read_page(contract)
return frontmatter
def authorised_labels(source: str, destination: str, kb_dir: Path | None = None) -> set[str]:
"""Labels a page in `source` may use on an edge into `destination`.
The union of the destination's own entry and `any`. An empty result means
the collection authorises nothing for that destination - which is a real
answer ("do not link there from here"), not a missing declaration.
"""
root = kb_dir if kb_dir is not None else config.KB_DIR
declared = collection_declaration(root / source).get(OUTBOUND_FIELD)
if not isinstance(declared, dict):
return set()
labels: set[str] = set()
for key in (destination, ANY_DESTINATION):
entry = declared.get(key)
if isinstance(entry, list):
labels.update(str(label).strip() for label in entry if str(label).strip())
return labels
def declaration_issues(kb_dir: Path | None = None) -> list[str]:
"""What each `COLLECTION.md` fails to declare about itself.
Two fields, for two questions the filesystem cannot answer. `profile:`
names the entry in `instructions/kb-profiles.md` this collection adopted -
free text, because the profile catalogue is a palette rather than an enum,
and a collection an instance invented has no entry there to name.
`required_by_stack:` is not the instance's to choose at all: it must agree
with what `stack_required_collections()` derives from the required types, so
a collection whose contract claims the stack depends on it - or one the
stack does depend on and that says it does not - is a finding rather than a
preference.
"""
root = kb_dir if kb_dir is not None else config.KB_DIR
issues: list[str] = []
required = stack_required_collections()
present = {path.name for path in iter_kb_collections(root)}
for name in required:
if name not in present:
issues.append(
f"kb/{name}/ is missing - it is where the stack-required `source` type writes, "
f"and `sources coverage`, `[^cite-id]` resolution and `kb/provenance.md` all "
f"depend on those pages existing"
)
for collection in iter_kb_collections(root):
relative = f"kb/{collection.name}/{CONTRACT_NAME}"
declared = collection_declaration(collection)
if not declared:
issues.append(
f"{relative} has no frontmatter - it must declare `{PROFILE_FIELD}:` and "
f"`{REQUIRED_BY_STACK_FIELD}:` (see instructions/kb-profiles.md)"
)
continue
profile = declared.get(PROFILE_FIELD)
if not isinstance(profile, str) or not profile.strip():
issues.append(
f"{relative}: `{PROFILE_FIELD}:` is missing or empty - name the profile from "
f"instructions/kb-profiles.md this collection adopted, or `none`"
)
required_flag = declared.get(REQUIRED_BY_STACK_FIELD)
expected = collection.name in required
if not isinstance(required_flag, bool):
issues.append(
f"{relative}: `{REQUIRED_BY_STACK_FIELD}:` is missing or not a boolean - "
f"it must be {str(expected).lower()} for this collection"
)
elif required_flag != expected:
issues.append(
f"{relative}: `{REQUIRED_BY_STACK_FIELD}: {str(required_flag).lower()}` "
f"contradicts the stack, which "
+ (
"does depend on this collection by name"
if expected
else "depends on no collection of this name"
)
+ f" - it must be {str(expected).lower()}"
)
return issues
def _is_vendored(path: Path, repo_root: Path) -> bool: def _is_vendored(path: Path, repo_root: Path) -> bool:
try: try:
relative = path.relative_to(repo_root) relative = path.relative_to(repo_root)
+3 -3
View File
@@ -14,9 +14,9 @@ WIKILINK_RE = re.compile(r"\[\[([^\]|#]+)")
# Root-level files under kb/ that are not pages: the generated catalog map, log # Root-level files under kb/ that are not pages: the generated catalog map, log
# and provenance index, plus the contract that constrains the tree rather than # and provenance index, plus the two documents that constrain the tree rather
# living in it. # than living in it - the stack's contract and this instance's own conventions.
_KB_META_FILES = {"index.md", "log.md", "provenance.md", "CONTRACT.md"} _KB_META_FILES = {"index.md", "log.md", "provenance.md", "CONTRACT.md", "CONVENTIONS.md"}
# The per-collection authoring contract. Unlike the meta files above it is never # The per-collection authoring contract. Unlike the meta files above it is never
# at the kb root - it sits one level down, in every collection - so it has to be # at the kb root - it sits one level down, in every collection - so it has to be
+101 -6
View File
@@ -39,6 +39,16 @@ def kb_state_file() -> Path:
return config.ROOT / KB_STATE_FILENAME return config.ROOT / KB_STATE_FILENAME
# Whether a migration has to run, as opposed to how it is carried out. The two
# are independent: a `mechanical` migration can be optional and an `assisted`
# one mandatory. Keeping them on one axis is what would make `migrate status`
# cry wolf - an instance nagged about an improvement it declined stops reading
# the nag that means its content no longer fits the machinery.
REQUIRED = "required"
OFFERED = "offered"
OBLIGATIONS = (REQUIRED, OFFERED)
@dataclass(frozen=True) @dataclass(frozen=True)
class Migration: class Migration:
"""One migration document under `instructions/migrations/`.""" """One migration document under `instructions/migrations/`."""
@@ -48,6 +58,11 @@ class Migration:
kind: str # "mechanical" | "assisted" kind: str # "mechanical" | "assisted"
description: str description: str
path: Path path: Path
obligation: str = REQUIRED
@property
def is_required(self) -> bool:
return self.obligation != OFFERED
@property @property
def relative_path(self) -> str: def relative_path(self) -> str:
@@ -130,6 +145,7 @@ def load_migrations() -> list[Migration]:
target = Version.parse(str(raw_target)) target = Version.parse(str(raw_target))
except VersionError: except VersionError:
continue continue
obligation = str(frontmatter.get("obligation") or REQUIRED)
migrations.append( migrations.append(
Migration( Migration(
name=str(frontmatter.get("name") or path.stem), name=str(frontmatter.get("name") or path.stem),
@@ -137,6 +153,7 @@ def load_migrations() -> list[Migration]:
kind=str(frontmatter.get("migration_kind") or "assisted"), kind=str(frontmatter.get("migration_kind") or "assisted"),
description=str(frontmatter.get("description") or ""), description=str(frontmatter.get("description") or ""),
path=path, path=path,
obligation=obligation if obligation in OBLIGATIONS else REQUIRED,
) )
) )
return sorted(migrations, key=lambda m: m.target) return sorted(migrations, key=lambda m: m.target)
@@ -147,13 +164,46 @@ def chain(
) -> list[Migration]: ) -> list[Migration]:
"""The migrations still owed, in the order they must run. """The migrations still owed, in the order they must run.
Every migration whose target lies in `(kb_version, stack_version]`, oldest Every **required** migration whose target lies in
first. An instance at 1.3.1 upgrading to 2.0.0 gets 1.4.0, 1.7.0, 2.0.0 - `(kb_version, stack_version]`, oldest first. An instance at 1.3.1 upgrading
and the absence of any migration targeting 1.3.x is not a special case, it to 2.0.0 gets 1.4.0, 1.7.0, 2.0.0 - and the absence of any migration
simply is not in the interval. Targets above the installed machinery are targeting 1.3.x is not a special case, it simply is not in the interval.
excluded: the instance has no code for them yet. Targets above the installed machinery are excluded: the instance has no code
for them yet.
`offered` migrations are deliberately absent. They are not links in the
version chain: declining one leaves the content in a shape the machinery
still accepts, so counting it as owed would make `kb_version` unreachable
for an instance that simply kept its own file.
""" """
return [m for m in migrations if kb_version < m.target <= stack_version] return [
m for m in migrations if m.is_required and kb_version < m.target <= stack_version
]
def applied_names(state: Optional[dict]) -> set[str]:
"""Every migration this instance has recorded as carried out."""
entries = (state or {}).get("applied") or []
return {
str(entry.get("migration"))
for entry in entries
if isinstance(entry, dict) and entry.get("migration")
}
def offers(migrations: list[Migration], applied: set[str]) -> list[Migration]:
"""Optional upgrades this instance has not taken, oldest target first.
Bounded by the **applied ledger**, not by `kb_version`, and that is not a
detail: taking an offer deliberately does not move `kb_version`, so the
version says nothing about whether an offer was taken. Filtering by it
would hide every offer the moment some unrelated required migration ran.
Not bounded above by the stack version either. An offer is about a file the
instance owns rather than about the shape of its content, so it stays on the
table until it is recorded - or until the operator deletes the document.
"""
return [m for m in migrations if not m.is_required and m.name not in applied]
def next_link( def next_link(
@@ -161,3 +211,48 @@ def next_link(
) -> Optional[Migration]: ) -> Optional[Migration]:
pending = chain(migrations, kb_version, stack_version) pending = chain(migrations, kb_version, stack_version)
return pending[0] if pending else None return pending[0] if pending else None
# --- what this instance changed about what it was given --------------------
def divergent_files() -> Optional[list[str]]:
"""Files whose content no longer matches the release this instance installed.
Reads the per-file sha256 in `.wikitool-release.json`, which `dist export`
has been writing since the stamp existed and which nothing has read until
now. Its own docstring says why it is there: it is the only way a later
upgrade can tell a file the instance *edited* from one it merely *received*.
That distinction is what makes an `offered` migration actionable. The stack
proposing a better `entity` template needs to know whether it may be copied
over or whether the instance has its own version that a person has to
reconcile - and only the recorded hash can answer that.
Returns None when the question is unanswerable (a development tree, which
carries no stamp), which is different from `[]` (nothing diverged).
"""
import hashlib
from chemenu import version as version_mod
try:
stamp = version_mod.read_stamp()
except VersionError:
return None
if not stamp:
return None
recorded = stamp.get("files")
if not isinstance(recorded, dict):
return None
divergent: list[str] = []
for relative, digest in sorted(recorded.items()):
path = config.ROOT / relative
if not path.is_file():
divergent.append(relative)
continue
current = "sha256:" + hashlib.sha256(path.read_bytes()).hexdigest()
if current != digest:
divergent.append(relative)
return divergent
+143
View File
@@ -0,0 +1,143 @@
"""Labelled edges in a page's `related:` frontmatter.
An edge is a **label plus a target**, and the label is an identifier rather than
prose:
related:
- depends-on: Hermes
- implements: Hybrid Search
It used to be a bare list of titles with the label written only into a body
bullet - which meant the graph's semantics lived in German prose the tool had to
parse back, and the vocabulary drifted to 152 distinct labels in 337 bullets
because nothing could check it. The label moves into the data; the body bullet
becomes a rendering of the data.
**Both shapes read.** A bare string is an edge whose label is not yet declared,
which is exactly the state a page is in between the machinery landing and the
corpus migration reaching that page. Readers therefore never crash on the old
shape, and `lint` is what reports it - the migration is finished when no
unlabelled edge is left.
Direction is authored, never mirrored: an edge lives on the page that asserts
it, and the inbound view is rendered from the graph rather than stored. See
instructions/link-taxonomy.md.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Iterable, Optional
# The label a not-yet-migrated bare-string edge reports as. Deliberately not a
# real catalogue label: it must be impossible for an instance to authorise it,
# so `lint` cannot be satisfied by declaring the placeholder legal.
UNLABELLED = None
@dataclass(frozen=True)
class Edge:
"""One declared relationship: what this page asserts about `target`."""
target: str
label: Optional[str] = UNLABELLED
@property
def is_labelled(self) -> bool:
return bool(self.label)
def parse_entry(entry: Any) -> Optional[Edge]:
"""One `related:` element as an Edge, or None if it is not one at all.
A single-key mapping is a labelled edge; a bare string is an unlabelled one.
Anything else - a multi-key mapping, a list, a number - is malformed, and
returning None rather than guessing is what lets `lint` report it as a
finding instead of a reader silently inventing an edge.
"""
if isinstance(entry, str):
title = entry.strip()
return Edge(title) if title else None
if isinstance(entry, dict) and len(entry) == 1:
(label, target), = entry.items()
label, target = str(label).strip(), str(target).strip()
return Edge(target, label) if label and target else None
return None
def edges(frontmatter: dict[str, Any], field: str) -> list[Edge]:
"""Every well-formed edge in `field`, in file order."""
parsed = (parse_entry(entry) for entry in (frontmatter.get(field) or []))
return [edge for edge in parsed if edge is not None]
def malformed(frontmatter: dict[str, Any], field: str) -> list[Any]:
"""Elements of `field` that are neither a title nor a `label: target` pair."""
return [
entry for entry in (frontmatter.get(field) or []) if parse_entry(entry) is None
]
def targets(frontmatter: dict[str, Any], field: str) -> list[str]:
"""Just the page titles in `field`, labelled or not.
The compatibility seam. Every caller that only ever wanted "which pages does
this reference" - dangling-reference checks, `rename`, `rm`, the link graph -
goes through here and is untouched by the label carried alongside.
"""
return [edge.target for edge in edges(frontmatter, field)]
def render(edge_list: Iterable[Edge]) -> list[Any]:
"""Edges back into frontmatter form, ready for `dump_frontmatter`.
An unlabelled edge round-trips as a bare string rather than being promoted
to some default label: inventing one here would erase the very thing `lint`
is looking for.
"""
rendered: list[Any] = []
for edge in edge_list:
rendered.append({edge.label: edge.target} if edge.is_labelled else edge.target)
return rendered
def upsert(frontmatter: dict[str, Any], field: str, edge: Edge) -> bool:
"""Add or relabel `edge` in `field`. True if anything changed.
Idempotent by target: one page asserts one thing about another, so a second
call with a different label *replaces* rather than appends. Two edges to the
same target would render two bullets and leave no way to say which is meant.
"""
current = edges(frontmatter, field)
for position, existing in enumerate(current):
if existing.target == edge.target:
if existing.label == edge.label:
return False
current[position] = edge
frontmatter[field] = render(current)
return True
current.append(edge)
frontmatter[field] = render(current)
return True
def remove(frontmatter: dict[str, Any], field: str, target: str) -> bool:
"""Drop every edge pointing at `target`. True if anything changed."""
current = edges(frontmatter, field)
kept = [edge for edge in current if edge.target != target]
if len(kept) == len(current):
return False
frontmatter[field] = render(kept)
return True
def retarget(frontmatter: dict[str, Any], field: str, old: str, new: str) -> bool:
"""Repoint every edge from `old` to `new`, keeping its label."""
current = edges(frontmatter, field)
changed = False
for position, edge in enumerate(current):
if edge.target == old:
current[position] = Edge(new, edge.label)
changed = True
if changed:
frontmatter[field] = render(current)
return changed
+515
View File
@@ -0,0 +1,515 @@
"""The lint core, with no CLI attached.
Split out of `commands/lint.py` for the reason given in
`chemenu/search/service.py`: `run_lint()` is a pure function over a corpus
directory, and it was sitting in a module that imports `typer` and `rich`, so
no in-process caller could reach it without the CLI head.
Everything that decides *findings* lives here. Everything that decides *how a
terminal sees them* - the report file, the exit code, the flags - stays in
`commands/lint.py`. `render_markdown()` and `render_summary()` are on this side
of the line because the markdown report is a data product (it is what
`reports/` holds and what `kb/log.md` refers to), not terminal formatting.
"""
from __future__ import annotations
from datetime import date
from pathlib import Path
from chemenu import blocks, config, kb_collections, links
from chemenu.frontmatter_io import frontmatter_error
from chemenu.markdown_code import strip_code_spans
from chemenu.provenance import broken_raw_refs as find_broken_raw_refs
from chemenu.provenance import duplicate_raw_file_owners as find_duplicate_raw_file_owners
from chemenu.provenance import extract_inline_cites
from chemenu.provenance import legacy_citation_markers as find_legacy_citation_markers
from chemenu.provenance import legacy_source_pages as find_legacy_source_pages
from chemenu.provenance import orphan_footnote_defs as find_orphan_footnote_defs
from chemenu.provenance import uncovered_raw_files as find_uncovered_raw_files
from chemenu.provenance import undefined_footnote_refs as find_undefined_footnote_refs
from chemenu.kb_scan import (
GENERATED_INDEX,
WIKILINK_RE,
build_link_graph,
find_duplicate_title_paths,
inbound_links,
load_kb_pages,
)
from chemenu.type_resolver import resolver
# Style guide's one mechanically-checkable rule (hard oracle: a plain count).
# The rest of the style guide (tone, AI-phrase avoidance) is a soft/proxy judgment
# and stays with the LLM - see wiki-manage/wiki-ingest skill guidance, not lint.
#
# The unit is a quote, not a `>` line. It used to be the line, which measured
# the wrap width the rule has no opinion about: one quotation written long
# counted 1 and the same quotation wrapped at 100 columns counted 4. An author
# who took the finding seriously made the page harder to read to quiet it.
QUOTE_LIMIT = 2
# How many hub pages `most_linked` reports. Purely informational (wiki-status
# surfaces it); not a finding, so the cutoff only bounds report size.
MOST_LINKED_COUNT = 10
def count_quote_blocks(body: str) -> int:
"""How many distinct blockquotes `body` carries.
A run of consecutive `>` lines is one quote; a blank line or any
non-quoted line ends it. Code is masked out first, so a `>` inside a
fenced shell transcript is a prompt, not a quotation.
Lazy continuation - a quote whose wrapped lines drop the `>` - reads here
as two quotes rather than one. That over-counts in the direction the limit
already errs on, and the corpus prefixes every line, so the alternative
(tracking paragraph state) buys nothing.
"""
count, in_quote = 0, False
for line in strip_code_spans(body).splitlines():
is_quote = line.lstrip().startswith(">")
if is_quote and not in_quote:
count += 1
in_quote = is_quote
return count
def run_lint(kb_dir: Path) -> dict:
pages = load_kb_pages(kb_dir)
duplicate_titles = find_duplicate_title_paths(kb_dir, config.ROOT)
# Pages whose frontmatter can't be parsed read back as `{}` everywhere
# else, which would let them slip past every frontmatter-driven check
# below with no finding at all - so they are detected explicitly.
frontmatter_errors = []
for title, page in sorted(pages.items()):
reason = frontmatter_error(page.path)
if reason is None and not page.frontmatter.get("type"):
reason = "missing `type:` field"
if reason is not None:
frontmatter_errors.append({"page": title, "error": reason})
graph = build_link_graph(pages)
broken_links = [
{"page": title, "target": target}
for title, targets in graph.items()
for target in sorted(targets)
if target not in pages
]
inbound = inbound_links({t: v for t, v in graph.items() if t != "index"})
orphan_pages = sorted(
title
for title, sources in inbound.items()
if not sources
and title not in ("index", "log")
# comparison pages are not linked to by design; index.md is sufficient coverage
and pages[title].kind != "comparison"
)
# Same link graph, opposite end: the most-linked-to pages are the wiki's
# hubs. Reported (not judged) so `wiki-status` can show them without
# re-deriving the graph.
inbound_counts = {title: len(sources) for title, sources in inbound.items()}
most_linked = [
{"page": title, "inbound": count}
for title, count in sorted(inbound_counts.items(), key=lambda kv: (-kv[1], kv[0]))
if count > 0
][:MOST_LINKED_COUNT]
# The catalog is sharded: `kb/index.md` is a map carrying counts and links,
# and the page rows live in a generated INDEX.md per collection/area. Both
# halves have to be read, or every page reads as missing from the index.
index_text = "".join(
path.read_text(encoding="utf-8")
for path in [kb_dir / "index.md", *sorted(kb_dir.rglob(GENERATED_INDEX))]
if path.exists()
)
index_links = {m.group(1).strip() for m in WIKILINK_RE.finditer(index_text)}
missing_from_index = sorted(set(pages) - index_links - {"index", "log"})
dangling_index_entries = sorted(index_links - set(pages))
title_mismatches = []
for title, page in sorted(pages.items()):
if page.kind not in ("entity", "concept"):
continue
h1 = page.h1_title
if h1 is not None and h1 != title:
title_mismatches.append({"page": title, "h1": h1})
unmarked_provenance = []
for title, page in sorted(pages.items()):
if page.kind not in ("entity", "concept"):
continue
sources_list = page.frontmatter.get("sources") or []
if not sources_list and page.frontmatter.get("provenance") != "general":
unmarked_provenance.append(title)
citation_frontmatter_drift = []
for title, page in sorted(pages.items()):
sources_list = set(page.frontmatter.get("sources") or [])
cited = {cited_title for cited_title, _file in extract_inline_cites(page.body)}
cited.discard(title) # a source page citing itself for a specific file within it is not drift
for missing_source in sorted(cited - sources_list):
citation_frontmatter_drift.append({"page": title, "cited_but_not_in_sources": missing_source})
legacy_citation_markers = find_legacy_citation_markers(pages)
undefined_footnote_refs = find_undefined_footnote_refs(pages)
orphan_footnote_defs = find_orphan_footnote_defs(pages)
# The frontmatter half of the link graph. `broken_links` above only walks
# `[[wikilinks]]` in page *bodies*, so a `related:`/`sources:`/`entities:`
# entry naming a page that does not exist - a rename that was not
# propagated, a deleted page, or a URL pasted where a title belongs - used
# to pass every check. Which fields hold page titles is declared by each
# type-spec's `page_ref_fields:`, not hardcoded here.
def _collection_of(page):
try:
return page.path.relative_to(config.KB_DIR).parts[0]
except (ValueError, IndexError):
return None
dangling_frontmatter_refs = []
malformed_edges: list[dict] = []
unlabelled_edges: list[dict] = []
unauthorised_labels: list[dict] = []
for title, page in sorted(pages.items()):
type_path = page.frontmatter.get("type")
if not type_path:
continue
try:
ref_fields = resolver.get_page_ref_fields(type_path, page.path)
except ValueError:
continue # unresolvable type is already reported as type_resolution_errors
for field in ref_fields:
# Through `links` so a labelled edge (`- depends-on: Hermes`) is read
# as its target rather than as a mapping - the entry carries the
# label alongside the title now, and comparing the whole entry would
# report every declared edge as dangling.
for target in links.targets(page.frontmatter, field):
if target not in pages:
dangling_frontmatter_refs.append(
{"page": title, "field": field, "target": target}
)
for entry in links.malformed(page.frontmatter, field):
malformed_edges.append(
{"page": title, "field": field, "entry": str(entry)}
)
# Labels are checked on `related:` only. `sources:`/`entities:`/
# `concepts:` are the provenance path, unlabelled by construction.
if "related" in ref_fields:
source_collection = _collection_of(page)
for edge in links.edges(page.frontmatter, "related"):
if not edge.is_labelled:
unlabelled_edges.append({"page": title, "target": edge.target})
continue
target_page = pages.get(edge.target)
if source_collection is None or target_page is None:
continue
destination = _collection_of(target_page)
if destination is None:
continue
allowed = kb_collections.authorised_labels(source_collection, destination)
if edge.label not in allowed:
unauthorised_labels.append(
{
"page": title,
"target": edge.target,
"label": edge.label,
"destination": destination,
}
)
# A generated region whose markers do not pair up is not a tidiness problem:
# the next write appends a second region beside it instead of replacing it,
# and the page then carries two. An agent rewriting prose at the boundary is
# how a marker goes missing, which is why this is a hard error.
unbalanced_marker_findings = [
{"page": title, "region": name}
for title, page in sorted(pages.items())
for name in blocks.unbalanced_markers(page.body)
]
quote_limit_violations = []
for title, page in sorted(pages.items()):
quote_count = count_quote_blocks(page.body)
if quote_count > QUOTE_LIMIT:
quote_limit_violations.append({"page": title, "quote_count": quote_count})
# Type system validation. Lint reports are not validated here: they are
# written to `reports/` outside kb/ and are never pages, so nothing this
# loop scans can be one.
invalid_type_paths = []
type_resolution_errors = []
schema_validation_errors = []
for title, page in sorted(pages.items()):
type_path = page.frontmatter.get("type")
if not type_path:
continue
# Check if type path is valid
if not type_path.endswith('.md'):
invalid_type_paths.append({"page": title, "type": type_path, "error": "Type path must end with .md"})
continue
# Try to resolve and validate the type
try:
resolver.load_type_spec(type_path, page.path)
# Try schema validation
try:
resolver.validate_frontmatter(page.frontmatter, type_path, page.path)
except ValueError as schema_error:
schema_validation_errors.append({"page": title, "type": type_path, "error": str(schema_error)})
except ValueError as resolution_error:
type_resolution_errors.append({"page": title, "type": type_path, "error": str(resolution_error)})
return {
"generated": date.today().isoformat(),
"page_count": len(pages),
"frontmatter_errors": frontmatter_errors,
"broken_links": broken_links,
"orphan_pages": orphan_pages,
"most_linked": most_linked,
"inbound_counts": inbound_counts,
"missing_from_index": missing_from_index,
"dangling_index_entries": dangling_index_entries,
"title_mismatches": title_mismatches,
"duplicate_titles": duplicate_titles,
"uncovered_raw_files": find_uncovered_raw_files(config.RAW_DIR, pages),
"broken_raw_refs": find_broken_raw_refs(pages),
"duplicate_raw_file_owners": find_duplicate_raw_file_owners(pages),
"legacy_source_pages": find_legacy_source_pages(pages),
"unmarked_provenance": unmarked_provenance,
"citation_frontmatter_drift": citation_frontmatter_drift,
"legacy_citation_markers": legacy_citation_markers,
"undefined_footnote_refs": undefined_footnote_refs,
"orphan_footnote_defs": orphan_footnote_defs,
"dangling_frontmatter_refs": dangling_frontmatter_refs,
"malformed_edges": malformed_edges,
"unlabelled_edges": unlabelled_edges,
"unauthorised_labels": unauthorised_labels,
"unbalanced_markers": unbalanced_marker_findings,
"quote_limit_violations": quote_limit_violations,
"invalid_type_paths": invalid_type_paths,
"type_resolution_errors": type_resolution_errors,
"schema_validation_errors": schema_validation_errors,
}
def _section(lines: list[str], title: str, items: list, formatter) -> None:
lines.append(f"## {title}")
lines.append("")
if not items:
lines.append("None found.")
else:
for item in items:
lines.append(f"- {formatter(item)}")
lines.append("")
def render_markdown(report: dict) -> str:
lines = [f"# Structural Lint Report ({report['generated']})", ""]
lines.append(f"Scanned {report['page_count']} pages under `wiki/`. This report covers only")
lines.append("mechanically-verifiable structural issues; see the Semantic Review section")
lines.append("below for judgment calls the LLM should complete.")
lines.append("")
_section(
lines, "Unreadable Frontmatter", report["frontmatter_errors"],
lambda i: f"[[{i['page']}]] - {i['error']}",
)
_section(
lines, "Broken Wikilinks", report["broken_links"],
lambda i: f"[[{i['page']}]] links to missing [[{i['target']}]]",
)
_section(lines, "Orphan Pages (no inbound links)", report["orphan_pages"], lambda i: f"[[{i}]]")
_section(
lines, f"Most-Linked Pages (top {MOST_LINKED_COUNT} hubs)", report["most_linked"],
lambda i: f"[[{i['page']}]] - {i['inbound']} inbound link(s)",
)
_section(lines, "Pages Missing from index.md", report["missing_from_index"], lambda i: f"[[{i}]]")
_section(lines, "Dangling index.md Entries", report["dangling_index_entries"], lambda i: f"[[{i}]]")
_section(
lines, "Duplicate Titles (naming collisions)", report["duplicate_titles"],
lambda i: f"`{i['stem']}` -> {', '.join(f'`{p}`' for p in i['paths'])}",
)
_section(
lines, "Filename / H1 Title Mismatches", report["title_mismatches"],
lambda i: f"[[{i['page']}]] H1 is '{i['h1']}'",
)
_section(
lines, "Uncovered Raw Files (no source page)", report["uncovered_raw_files"],
lambda i: f"`{i}`",
)
_section(
lines, "Broken raw_files References", report["broken_raw_refs"],
lambda i: f"[[{i['page']}]] -> `{i['raw_path']}` (does not exist)",
)
_section(
lines, "Raw Files With More Than One Owner", report["duplicate_raw_file_owners"],
lambda i: f"`{i['raw_file']}` is claimed by " + ", ".join(f"[[{t}]]" for t in i["owners"]),
)
_section(
lines, "Legacy source: Field (not yet migrated to raw_files:)", report["legacy_source_pages"],
lambda i: f"[[{i['page']}]] source: `{i['source']}` ({i['reason']})",
)
_section(
lines, "Pages Missing provenance: general Marker", report["unmarked_provenance"],
lambda i: f"[[{i}]] has no sources and is not marked `provenance: general`",
)
_section(
lines, "Citation / Frontmatter Drift", report["citation_frontmatter_drift"],
lambda i: f"[[{i['page']}]] cites [[{i['cited_but_not_in_sources']}]] inline but it is missing from frontmatter `sources:`",
)
_section(
lines, "Legacy Citation Markers (pre-migration `^[[...]]`)", report["legacy_citation_markers"],
lambda i: f"[[{i['page']}]] still has `{i['marker']}` - run `wikitool cite add` and replace it with the `[^cite-id]` it prints",
)
_section(
lines, "Undefined Footnote References", report["undefined_footnote_refs"],
lambda i: f"[[{i['page']}]] references `[^{i['ref']}]`, which has no `[^{i['ref']}]: [[...]]` definition",
)
_section(
lines, "Orphan Footnote Definitions", report["orphan_footnote_defs"],
lambda i: f"[[{i['page']}]] defines `[^{i['id']}]` (-> [[{i['source']}]]) but nothing references it - run `wikitool cite sync`",
)
_section(
lines, "Malformed Edges", report.get("malformed_edges", []),
lambda i: f"[[{i['page']}]] `{i['field']}`: {i['entry']}",
)
_section(
lines, "Unbalanced Generated-Region Markers", report.get("unbalanced_markers", []),
lambda i: f"[[{i['page']}]]: `{i['region']}`",
)
_section(
lines, "Unlabelled Edges", report.get("unlabelled_edges", []),
lambda i: f"[[{i['page']}]] -> [[{i['target']}]]",
)
_section(
lines, "Labels Not Authorised by the Source Collection",
report.get("unauthorised_labels", []),
lambda i: f"[[{i['page']}]] `{i['label']}` -> kb/{i['destination']}/ ([[{i['target']}]])",
)
_section(
lines, "Dangling Frontmatter References", report["dangling_frontmatter_refs"],
lambda i: f"[[{i['page']}]] `{i['field']}:` names `{i['target']}`, which is not a page",
)
_section(
lines, "Invalid Type Paths", report["invalid_type_paths"],
lambda i: f"[[{i['page']}]] has type: `{i['type']}` - {i['error']}",
)
_section(
lines, "Type Resolution Errors", report["type_resolution_errors"],
lambda i: f"[[{i['page']}]] type: `{i['type']}` - {i['error']}",
)
_section(
lines, "Schema Validation Errors", report["schema_validation_errors"],
lambda i: f"[[{i['page']}]] type: `{i['type']}` - {i['error']}",
)
_section(
lines, f"Pages Exceeding Quote Limit (>{QUOTE_LIMIT}/page)", report["quote_limit_violations"],
lambda i: f"[[{i['page']}]] has {i['quote_count']} quotes - trim or confirm they're load-bearing",
)
lines.append("## Semantic Review (LLM to complete)")
lines.append("")
lines.append("- Contradictions across pages: TODO")
lines.append("- Stale claims (unconfirmed >6 months): TODO")
lines.append("- Suggested new pages / missing cross-references: TODO")
lines.append("")
return "\n".join(lines)
# Sections that always carry content but are not findings, so the summary
# handles them separately: a hub list is a statistic, and the semantic review
# is the checklist that follows the report rather than part of it.
INFORMATIONAL_SECTIONS = ("Most-Linked Pages",)
SEMANTIC_REVIEW_SECTION = "Semantic Review"
def _split_sections(markdown: str) -> tuple[str, list[tuple[str, str]]]:
"""Cut a rendered report into its preamble and (title, body) sections."""
preamble, *rest = markdown.split("\n## ")
sections = []
for part in rest:
title, _, body = part.partition("\n")
sections.append((title.strip(), body.strip()))
return preamble.rstrip(), sections
def render_summary(report: dict) -> str:
"""The same report with the empty sections removed.
On a healthy corpus the full report is better than 90% "None found.", so
reading it in the terminal means paging past the answer. The file on disk
stays complete - this is what gets printed, and the written path underneath
it is how the rest is reached without running lint a second time.
"""
preamble, sections = _split_sections(render_markdown(report))
findings, trailing = [], []
for title, body in sections:
if title.startswith(SEMANTIC_REVIEW_SECTION):
trailing.append((title, body))
elif body != "None found." and not title.startswith(INFORMATIONAL_SECTIONS):
findings.append((title, body))
lines = [preamble, ""]
if not findings:
lines += ["No structural findings.", ""]
for title, body in findings + trailing:
lines += [f"## {title}", "", body, ""]
return "\n".join(lines)
def default_report_path(report: dict) -> Path:
"""Where a report goes when the caller names no path.
`reports/` is derived and gitignored ([reports/CONTRACT.md]), so writing
here by default costs the tree nothing.
"""
return config.ROOT / "reports" / f"Lint Report {report['generated']}.md"
# Findings that make a tree structurally wrong rather than merely untidy.
# `orphan_pages` is deliberately absent: many pages are validly reachable
# through the index or navigation only. `quote_limit_violations` is advisory
# too - it flags a habit, not a broken tree.
#
# `unlabelled_edges` and `unauthorised_labels` are advisory **for now**, and
# that is a dated decision rather than a judgment about severity: they describe
# exactly the state a corpus is in between the 4.0.0 machinery landing and the
# migration reaching each page, which is the window `.wikitool-kb.json` exists
# to represent. They become hard errors once the migration is recorded - the
# same path `legacy_citation_markers` took.
#
# `malformed_edges` and `unbalanced_markers` are hard from the start: neither
# describes an unconverted page, only a broken one.
#
# One definition, used by `lint --fail-on-error` and by the eval scorecard: if
# the two disagreed, a run could pass its score while lint refused it.
HARD_ERROR_KEYS = (
"frontmatter_errors",
"broken_links",
"dangling_index_entries",
"duplicate_titles",
"broken_raw_refs",
"duplicate_raw_file_owners",
"legacy_source_pages",
"citation_frontmatter_drift",
"legacy_citation_markers",
"undefined_footnote_refs",
"orphan_footnote_defs",
"dangling_frontmatter_refs",
"malformed_edges",
"unbalanced_markers",
"invalid_type_paths",
"type_resolution_errors",
"schema_validation_errors",
)
def has_hard_errors(report: dict) -> bool:
return any(report.get(key) for key in HARD_ERROR_KEYS)
+8
View File
@@ -0,0 +1,8 @@
"""The MCP read server: Chemenu's second consumer.
The CLI and this server are two adapters over one core (`chemenu.api`), not a
CLI with a network interface bolted on. See `server.py`.
"""
from chemenu.mcp.server import build_server
__all__ = ["build_server"]
+79
View File
@@ -0,0 +1,79 @@
"""`python -m chemenu.mcp` - start the read server.
Deliberately argparse and not typer. This process is the one place that must
not pull the CLI head in: the whole point of the library boundary is that a
second consumer costs `yaml`, `jsonschema` and the MCP SDK, and nothing else.
python -m chemenu.mcp # stdio
python -m chemenu.mcp --transport streamable-http # behind the proxy
CHEMENU_ROOT=/srv/wiki python -m chemenu.mcp --transport streamable-http \
--host 0.0.0.0 --port 8000
`--host`/`--port` apply to `streamable-http` only. They are here because the
default binds loopback, and a server in a container with a reverse proxy in
front of it has to bind an interface the proxy can reach - that is a property
of the software, not of one installation. *Which* host and port a given
deployment picks is infrastructure and stays out of this repository.
"""
from __future__ import annotations
import argparse
import sys
from chemenu.mcp.server import TRANSPORTS, TraceWouldWriteIntoCorpus, serve
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
prog="python -m chemenu.mcp",
description="Serve a Chemenu wiki read-only over MCP.",
)
parser.add_argument(
"--transport",
choices=TRANSPORTS,
default="stdio",
help="stdio for local use and testing; streamable-http for a deployed "
"instance behind the Traefik middleware (default: stdio)",
)
parser.add_argument(
"--root",
default=None,
help="The corpus to serve. Defaults to $CHEMENU_ROOT, then the checkout "
"this package lives in.",
)
parser.add_argument(
"--host",
default="127.0.0.1",
help="Interface to bind, streamable-http only. The loopback default is "
"deliberate; a container behind a reverse proxy needs 0.0.0.0.",
)
parser.add_argument(
"--port",
type=int,
default=8000,
help="Port to bind, streamable-http only (default: 8000)",
)
args = parser.parse_args(argv)
# Passed only for the transport they mean anything to: `run_stdio_async`
# takes no host or port, and handing it one is a TypeError rather than a
# harmless no-op.
bind = (
{"host": args.host, "port": args.port}
if args.transport == "streamable-http"
else {}
)
try:
serve(transport=args.transport, root=args.root, **bind)
except TraceWouldWriteIntoCorpus as exc:
# Refused before binding anything: the message names both fixes, and a
# server that silently relocated the operator's telemetry instead would
# be a surprise buried in a log.
print(f"ERROR {exc}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__": # pragma: no cover - process entry point
raise SystemExit(main())
+234
View File
@@ -0,0 +1,234 @@
"""The MCP read server: `search`, `types`, `lint` and `status` over `kb/`.
Chemenu's second consumer. The CLI and this are two adapters over one core -
`chemenu.api.Corpus` - so a question answered here and the same question asked
at a terminal go through the same code, and a golden test holds the two
outputs against each other rather than trusting that they agree.
**There is no write path, structurally.** Nothing under `chemenu.commands` is
imported here or in `chemenu.api`, so `new`, `touch`, `xref`, `cite`,
`publish`, `migrate` and `version bump` are not reachable - the functions do
not exist in this process's reach, rather than being filtered out of a list. A
test asserts it by importing this module in a clean interpreter and looking at
`sys.modules`.
**Authentication and rate limiting are not here.** Both are Traefik middleware
in front of the process, per the operator's decision of 2026-09-01: a request
that is not cleanly authenticated does not reach Python at all. What *is* here
is the resource protection that middleware cannot give - the search timeout and
the frontmatter limits - because those exist against an authenticated consumer
damaging itself, which is a different problem from an unauthenticated one.
**The Iteration Budget Gate is deliberately absent.** It exists to stop an
agent *session* from iterating unnoticed over the state of the wiki, which is
why retrieval is exempt from it in the first place. A user who searches too
often is a resource problem - different instrument, different purpose - and
using the gate as a rate limiter would dilute it into one.
**Every response carries the commit it was computed from.** `chemenu.api`
stamps `commit` and `as_of`; a stale checkout otherwise answers confidently and
wrongly. Keeping the checkout current is a `git fetch && git reset --hard`
poll outside this process - see `instructions/mcp-read-server.md` - which needs
no inbound endpoint and no signature checking.
"""
from __future__ import annotations
import os
from pathlib import Path
from typing import Any, Optional
from mcp.server.mcpserver import MCPServer
from mcp.server.mcpserver.exceptions import ToolError
from chemenu import config
from chemenu.api import Corpus
from chemenu.errors import ChemenuError
SERVER_NAME = "chemenu"
# Transports this server will start on. `stdio` is for developing and testing
# it without a network; `streamable-http` is what a deployed instance speaks,
# and the only one the Traefik middleware can sit in front of, because Traefik
# is an HTTP reverse proxy. `sse` is reachable through the SDK but not offered:
# it is the superseded remote transport, and building on it now only moves the
# migration later.
TRANSPORTS = ("stdio", "streamable-http")
class TraceWouldWriteIntoCorpus(RuntimeError):
"""Raised at startup when telemetry would land inside the served tree."""
def check_trace_destination(root: Path) -> None:
"""Refuse to start if a trace would be written into the corpus.
Telemetry defaults to *on* and writes under `reports/telemetry/` in the
repo. Today nothing on this path emits - the writer is wired into
`cli.main()` and the two gates, none of which run here - so this is a guard
against the future rather than a fix for the present. It is worth having
anyway: the sync that keeps this checkout current is `git reset --hard`, so
a trace written into the tree is both a per-request write into a directory
something else is entitled to wipe, and a silent way for the server to
dirty the tree its own cache keys on.
Turn tracing off (`WIKI_TRACE=0`) or point it somewhere else
(`WIKI_TRACE_DIR`). Refusing rather than correcting it: a server that
quietly relocates the operator's telemetry is a surprise waiting in a log
nobody reads.
"""
if os.environ.get("WIKI_TRACE", "1") == "0":
return
destination = os.environ.get("WIKI_TRACE_DIR")
if destination is None:
raise TraceWouldWriteIntoCorpus(
"Telemetry is on and would write into the served checkout "
f"({root / 'reports' / 'telemetry'}). The sync that keeps this checkout "
"current is `git reset --hard`, which is entitled to wipe that directory. "
"Set WIKI_TRACE=0, or point WIKI_TRACE_DIR outside the corpus."
)
resolved = Path(destination).expanduser().resolve()
try:
resolved.relative_to(Path(root).resolve())
except ValueError:
return
raise TraceWouldWriteIntoCorpus(
f"WIKI_TRACE_DIR ({resolved}) is inside the served checkout ({root}). "
"Point it outside, or set WIKI_TRACE=0."
)
def build_server(
root: Optional[Path | str] = None, check_trace: bool = True
) -> MCPServer:
"""Assemble the server over one corpus.
`root` follows `config.resolve_root()` - argument, then `$CHEMENU_ROOT`,
then the checkout the package lives in - so a deployment points at its
corpus with one environment variable and no code.
"""
corpus = Corpus(root)
if check_trace:
check_trace_destination(corpus.root)
server = MCPServer(
name=SERVER_NAME,
instructions=(
"Read access to a Chemenu wiki: compiled, sourced knowledge under kb/. "
"Every answer carries the commit it was computed from ('commit') and "
"when it was produced ('as_of'); a null commit means the served tree "
"has uncommitted changes and the answer corresponds to no revision. "
"This server is read-only - there is no tool that writes."
),
)
@server.tool(
name="search",
description=(
"Find pages in kb/ by text, by frontmatter, or by both. Returns "
"title, path, kind, summary and confidence per hit, so a result can "
"be judged without fetching the page. Prefer this over listing "
"files: the answer is a few hundred tokens instead of a whole index."
),
)
def search(
query: str | None = None,
predicates: list[str] | None = None,
regex: bool = False,
limit: int = 20,
sort: str | None = None,
) -> dict[str, Any]:
"""Search the wiki.
`predicates` are frontmatter filters in the CLI's own `--field` syntax,
ANDed: `confidence<0.6`, `entity_type=system`, `tags~k8s`, `source_url:*`
(present), `!source_url` (absent). With no `query` this is a pure
structured query over frontmatter.
`regex` applies the pattern with ripgrep's linear engine. It is off by
default, so an accidental `.*` is a literal.
"""
return _guard(
lambda: corpus.search(
text=query,
predicates=predicates or (),
regex=regex,
limit=limit,
sort=sort,
)
)
@server.tool(
name="types",
description=(
"List the page types this wiki declares - what kinds of page exist, "
"where each lives, and what its schema is. Read this before "
"interpreting a page's `kind`."
),
)
def types() -> dict[str, Any]:
return _guard(corpus.types)
@server.tool(
name="describe_type",
description=(
"One page type's full contract: its frontmatter fields with "
"required/optional and any enums, its subtype field, and its "
"authoring guidance."
),
)
def describe_type(name: str) -> dict[str, Any]:
"""`name` is a short type name as listed by `types`, e.g. 'entity'."""
return _guard(lambda: corpus.describe_type(name))
@server.tool(
name="lint",
description=(
"The wiki's structural health: broken wikilinks, orphan pages, "
"index drift, schema gaps, provenance gaps. Findings only - the "
"JSON form writes no report file."
),
)
def lint() -> dict[str, Any]:
return _guard(corpus.lint)
@server.tool(
name="status",
description=(
"A snapshot: how many pages the wiki holds, how they split across "
"collections, and how many findings of each kind lint reports. "
"Cheaper to read than the full lint output."
),
)
def status() -> dict[str, Any]:
return _guard(corpus.status)
return server
def _guard(call):
"""Turn a `ChemenuError` into a plain message for the protocol layer.
A bad predicate is the caller's argument, not a server fault, and it should
arrive as a tool error the model can act on, carrying the message that says
what to do differently. The SDK draws exactly this line: a `ToolError` is a
deliberate refusal and its text reaches the caller, while anything else is a
crash whose text stays on the server. Only `ChemenuError` is caught -
everything else is a genuine fault and belongs in the log, unswallowed.
"""
try:
return call()
except ChemenuError as exc:
raise ToolError(str(exc)) from exc
def serve(
transport: str = "stdio",
root: Optional[Path | str] = None,
**kwargs: Any,
) -> None:
"""Start the server. `transport` is one of `TRANSPORTS`."""
if transport not in TRANSPORTS:
raise ValueError(
f"unknown transport {transport!r}. Available: {', '.join(TRANSPORTS)}"
)
build_server(root).run(transport=transport, **kwargs)
+8
View File
@@ -17,6 +17,14 @@ class Page:
frontmatter: dict[str, Any] = field(default_factory=dict) frontmatter: dict[str, Any] = field(default_factory=dict)
body: str = "" body: str = ""
# Why the frontmatter above is empty, when it is empty for a reason. A page
# whose YAML does not parse reads back as `{}`, and a `{}` page has no
# `confidence` and no `kind`: it then drops out of `--field confidence<0.6`
# - the query whose whole purpose is to find pages in bad shape - looking
# exactly like a page that did not match. Loaders that know the reason put
# it here so a caller can report the page instead of losing it.
frontmatter_error: Optional[str] = None
@property @property
def title(self) -> str: def title(self) -> str:
"""The page's canonical title: the filename without extension. """The page's canonical title: the filename without extension.
+108 -79
View File
@@ -20,7 +20,7 @@ import unicodedata
from pathlib import Path from pathlib import Path
from typing import Optional from typing import Optional
from chemenu import config, sections from chemenu import blocks, config, conventions
from chemenu.markdown_code import strip_code_spans from chemenu.markdown_code import strip_code_spans
from chemenu.page import Page from chemenu.page import Page
@@ -45,10 +45,6 @@ CITE_DEF_RE = re.compile(
) )
CITE_REF_RE = re.compile(rf"\[\^({_CITE_ID_PATTERN})\]") CITE_REF_RE = re.compile(rf"\[\^({_CITE_ID_PATTERN})\]")
# Where the Footnotes block stops: the next ATX heading of any level. Without
# this the block ran to the end of the file and took any following section with
# it - see split_cite_block().
_NEXT_HEADING_RE = re.compile(r"^#{1,6} ", re.MULTILINE)
# The pre-migration marker: `^[[Source - X]]` or `^[[Source - X|file.md]]`, # The pre-migration marker: `^[[Source - X]]` or `^[[Source - X|file.md]]`,
# read by a Pandoc-style parser as an inline footnote wrapping a broken # read by a Pandoc-style parser as an inline footnote wrapping a broken
@@ -61,12 +57,29 @@ LEGACY_CITE_RE = re.compile(r"\^\[\[([^\]|#]+)(?:\|([^\]]+))?\]\]")
# footnote definitions regardless of the heading text; this heading is purely # footnote definitions regardless of the heading text; this heading is purely
# for human readability when the raw markdown is read directly. # for human readability when the raw markdown is read directly.
# #
# Written under the canonical name, but split_cite_block() matches the aliases # The prefix a source page's title carries, stripped when minting a cite id so
# too - a page whose block still says "## Footnotes" keeps working until it is # the id is not "s-source-x". It is the `source` type-spec's own
# translated. See chemenu/sections.py. # `title_prefix:`, asked for at call time rather than written down here: the
CITE_BLOCK_HEADING = f"## {sections.FOOTNOTES}" # type-spec belongs to the instance, so hardcoding the string made a documented
# instance decision into a compiler constant - the same leak `sections.py` had.
#
# The literal survives as the fallback for a tree with no resolvable `source`
# type (a fixture, a half-built instance). It is what this stack shipped, so a
# corpus that can reach the fallback was minted under it, and ids stay stable.
_FALLBACK_SOURCE_TITLE_PREFIX = "Source - "
_SOURCE_TITLE_PREFIX = "Source - "
def source_title_prefix() -> str:
"""This instance's source-page title prefix, from the type-spec."""
from chemenu.type_resolver import resolver
try:
type_path = resolver.find_type_by_name("source")
if type_path:
return resolver.get_title_prefix(type_path)
except (ValueError, OSError):
pass
return _FALLBACK_SOURCE_TITLE_PREFIX
@@ -102,7 +115,8 @@ def cite_id(title: str, qualifier: Optional[str] = None) -> str:
NFKD transliteration is lossy), so callers resolving a real page use NFKD transliteration is lossy), so callers resolving a real page use
unique_cite_id() to add a `-2`/`-3` suffix on collision. unique_cite_id() to add a `-2`/`-3` suffix on collision.
""" """
base_title = title[len(_SOURCE_TITLE_PREFIX):] if title.startswith(_SOURCE_TITLE_PREFIX) else title prefix = source_title_prefix()
base_title = title[len(prefix):] if prefix and title.startswith(prefix) else title
slug = "s-" + _slugify(base_title) slug = "s-" + _slugify(base_title)
if qualifier: if qualifier:
slug += "--" + _slugify(qualifier) slug += "--" + _slugify(qualifier)
@@ -123,62 +137,64 @@ def unique_cite_id(existing_ids: set[str], title: str, qualifier: Optional[str]
return f"{base}-{suffix}" return f"{base}-{suffix}"
def split_cite_block(body: str) -> tuple[str, dict[str, tuple[str, Optional[str]]]]: # Headings a pre-4.0.0 page carries above its citation definitions, for the
"""Split the Footnotes block off `body`. # migration window only. Before the block was delimited it was *located* by this
# text, which is why there are four of them - two languages times two eras. The
# list is read, never written, and `instructions/migrations/` removes the need
# for it once every page carries markers.
_LEGACY_FOOTNOTE_HEADINGS = ("Fußnoten", "Footnotes", "Fussnoten", "Notes")
Returns (body_without_block, definitions), where definitions maps _LEGACY_HEADING_RE = re.compile(
cite_id -> (source_title, qualifier_or_None) in file order. If there is r"^## (?:" + "|".join(re.escape(name) for name in _LEGACY_FOOTNOTE_HEADINGS) + r")[ \t]*$",
no Footnotes block, definitions is {} and body is returned with trailing re.MULTILINE,
blank lines trimmed (so re-rendering after emptying the block is stable). )
_NEXT_HEADING_RE = re.compile(r"^#{1,6} ", re.MULTILINE)
**The block is not "everything to the end of the file".** It used to be,
and every caller here reassembles a page as `head + rendered block` - so a
section that happened to sit after the block was silently deleted on the
next `cite add`, `cite sync` or `rename`. That is not hypothetical: `xref
add` appends its Relationships and See Also sections at the end of the
file, so whether a page kept its cross-references came down to which of the
two commands ran last. Eight pages were carrying content in that position
when this was found.
So the block ends where the next heading begins, and everything after it - def _definitions_in(block: str) -> dict[str, tuple[str, Optional[str]]]:
plus anything inside it that is not a citation definition - is folded back """Every `[^id]: [[Target]]` definition in one region, code masked out.
on to `head`. Nothing is discarded, and because the rendered block is
always emitted last, a page that had drifted into the broken layout is A fenced example of a definition line is an illustration, not a definition.
normalised the first time any of these commands touches it. `strip_code_spans` preserves offsets and line structure, so the masked text
reads line-for-line against the real one.
""" """
# Where the block *starts* is decided on the unmasked body, deliberately. masked = strip_code_spans(block)
# Masking first would mean one unclosed fence anywhere in the prose blanks return {
# the real `## Footnotes` heading too, and the page then reads as having no m.group(1): (m.group(2).strip(), m.group(3).strip() if m.group(3) else None)
# definitions at all - every citation on it undefined, from a single typo. for m in CITE_DEF_RE.finditer(masked)
# A fenced example of the heading itself is the rarer accident and the }
# cheaper one: it costs one page its block, not every citation on it.
match = sections.heading_re(sections.FOOTNOTES).search(body)
def _split_legacy_block(body: str) -> tuple[str, dict[str, tuple[str, Optional[str]]]]:
"""The pre-marker layout: a heading, then definitions, ending at the next
heading.
Kept only so the corpus stays readable between this machinery landing and
the migration reaching each page. Every weakness of the old approach lives
here - it guesses the region's end, and it can be fooled by a fenced example
of the heading - which is the argument the marker pair settles.
"""
match = _LEGACY_HEADING_RE.search(body)
if not match: if not match:
return body.rstrip("\n"), {} return body.rstrip("\n"), {}
head, rest = body[: match.start()], body[match.end():] head, rest = body[: match.start()], body[match.end():]
next_section = _NEXT_HEADING_RE.search(rest) following = _NEXT_HEADING_RE.search(rest)
block, trailing = (rest[: next_section.start()], rest[next_section.start():]) if next_section else (rest, "") block, trailing = (
(rest[: following.start()], rest[following.start():]) if following else (rest, "")
)
# Inside the block, code is masked: a fenced example of a definition line is definitions = _definitions_in(block)
# an illustration, not a definition. strip_code_spans() preserves offsets
# and line structure, so the masked block can be read line-for-line against
# the real one.
masked_block = strip_code_spans(block) masked_block = strip_code_spans(block)
definitions = {
m.group(1): (m.group(2).strip(), m.group(3).strip() if m.group(3) else None)
for m in CITE_DEF_RE.finditer(masked_block)
}
# Lines inside the block that are not definitions are content too - prose # Lines inside the block that are not definitions are content too - prose
# someone left there, a stray bullet. Rescued rather than rejected: this # someone left there, a stray bullet. Rescued rather than rejected: this runs
# runs under `lint` and `corpus_diff` as well, where raising would refuse # under `lint` and `corpus_diff` as well, where raising would refuse to read
# to read a page instead of reporting it. # a page instead of reporting it.
stray = "\n".join( stray = "\n".join(
line line
for line, masked in zip(block.splitlines(), masked_block.splitlines()) for line, masked in zip(block.splitlines(), masked_block.splitlines())
if line.strip() and not CITE_DEF_RE.match(masked) if line.strip() and not CITE_DEF_RE.match(masked)
) )
rescued = "\n\n".join(part.strip("\n") for part in (stray, trailing) if part.strip()) rescued = "\n\n".join(part.strip("\n") for part in (stray, trailing) if part.strip())
head = head.rstrip("\n") head = head.rstrip("\n")
if rescued: if rescued:
@@ -186,44 +202,57 @@ def split_cite_block(body: str) -> tuple[str, dict[str, tuple[str, Optional[str]
return head, definitions return head, definitions
def cite_block_heading(body: str) -> str: def split_cite_block(body: str) -> tuple[str, dict[str, tuple[str, Optional[str]]]]:
"""The Footnotes heading `body` actually carries, canonical if it has none. """Split the citation region off `body`.
Rewriting a page must not silently retitle its block: a page still using an Returns (body_without_region, definitions), where definitions maps
alias is untranslated, not broken, and `cite sync` has to stay a no-op on cite_id -> (source_title, qualifier_or_None) in file order.
it. Translating the heading is the migration's job, not the tool's."""
match = sections.heading_re(sections.FOOTNOTES).search(body) **The region is delimited, not guessed.** It used to end "at the next
return match.group(0).strip() if match else CITE_BLOCK_HEADING heading", and before that "at the end of the file" - and every caller here
reassembles a page as `head + rendered region`, so a section that happened to
sit after it was silently deleted on the next `cite add`, `cite sync` or
`rename`. Eight pages were carrying content in that position when it was
found. A marker pair answers where the region stops exactly, which is the
whole reason for it.
A page with no markers is read through the legacy path instead, so the
corpus stays readable until the migration reaches it.
"""
region = blocks.find(body, blocks.FOOTNOTES)
if region is None:
return _split_legacy_block(body)
return blocks.strip(body, blocks.FOOTNOTES).rstrip("\n"), _definitions_in(region)
def render_cite_block( def render_cite_block(definitions: dict[str, tuple[str, Optional[str]]]) -> str:
definitions: dict[str, tuple[str, Optional[str]]], heading: str = CITE_BLOCK_HEADING """The citation region for `definitions`, markers included, in dict order.
) -> str:
"""Render the Footnotes block for `definitions` (cite_id -> (title, An empty dict renders "" - a page with no citations carries no region at
qualifier)), preserving dict order. Empty dict renders "" - a page with all, rather than a heading with nothing under it.
no citations carries no block at all.""" """
if not definitions: lines = []
return ""
lines = [heading, ""]
for cid, (title, qualifier) in definitions.items(): for cid, (title, qualifier) in definitions.items():
target = f"{title}|{qualifier}" if qualifier else title target = f"{title}|{qualifier}" if qualifier else title
lines.append(f"[^{cid}]: [[{target}]]") lines.append(f"[^{cid}]: [[{target}]]")
return "\n".join(lines) + "\n" return blocks.render(
blocks.FOOTNOTES, conventions.heading(blocks.FOOTNOTES), lines
)
def render_page_body( def render_page_body(
head: str, head: str, definitions: dict[str, tuple[str, Optional[str]]]
definitions: dict[str, tuple[str, Optional[str]]],
heading: str = CITE_BLOCK_HEADING,
) -> str: ) -> str:
"""Reassemble a page body from its non-Footnotes content and citation """Reassemble a page body from its non-citation content and its definitions -
definitions - the inverse of split_cite_block(). Pass the original body's the inverse of `split_cite_block`.
`cite_block_heading()` to preserve an alias the page still uses."""
head = head.rstrip("\n") The heading is no longer threaded through from the caller. It used to be, so
block = render_cite_block(definitions, heading) that rewriting a page would not silently retitle a block whose text the tool
if not block: was *matching on*; now the marker carries the identity and the heading is a
return head + "\n" rendering value, so re-rendering it under this instance's own words is a
return head + "\n\n" + block repair rather than a rename.
"""
return blocks.replace(head.rstrip("\n") + "\n", blocks.FOOTNOTES, render_cite_block(definitions))
def extract_inline_cites(body: str) -> set[tuple[str, Optional[str]]]: def extract_inline_cites(body: str) -> set[tuple[str, Optional[str]]]:
+2 -1
View File
@@ -14,6 +14,7 @@ from pathlib import Path
from typing import Any, Optional from typing import Any, Optional
from chemenu import config from chemenu import config
from chemenu.errors import ValidationError
from chemenu.page import Page from chemenu.page import Page
from chemenu.search.types import Predicate from chemenu.search.types import Predicate
@@ -26,7 +27,7 @@ _COMPARISON_OPS = (">=", "<=", ">", "<", "~", "=")
VIRTUAL_FIELDS = ("title", "kind", "subtype", "collection") VIRTUAL_FIELDS = ("title", "kind", "subtype", "collection")
class PredicateError(ValueError): class PredicateError(ValidationError):
"""Raised for a malformed `--field` argument or an unknown field name.""" """Raised for a malformed `--field` argument or an unknown field name."""
+16 -5
View File
@@ -7,28 +7,39 @@ the output shape. Selecting several at once fuses them through RRF.
from __future__ import annotations from __future__ import annotations
import os import os
from typing import Callable from pathlib import Path
from typing import Callable, Optional
from chemenu.errors import ValidationError
from chemenu.search.base import SearchBackend from chemenu.search.base import SearchBackend
from chemenu.search.ripgrep import RipgrepBackend from chemenu.search.ripgrep import RipgrepBackend
DEFAULT_BACKEND = "rg" DEFAULT_BACKEND = "rg"
ENV_VAR = "WIKITOOL_SEARCH_BACKEND" ENV_VAR = "WIKITOOL_SEARCH_BACKEND"
BACKENDS: dict[str, Callable[[], SearchBackend]] = { BACKENDS: dict[str, Callable[..., SearchBackend]] = {
"rg": RipgrepBackend, "rg": RipgrepBackend,
} }
class UnknownBackend(ValueError): class UnknownBackend(ValidationError):
pass pass
def resolve(spec: str | None = None) -> list[SearchBackend]: def resolve(
spec: str | None = None,
kb_dir: Optional[Path] = None,
root: Optional[Path] = None,
) -> list[SearchBackend]:
"""Resolve a backend spec into instances. """Resolve a backend spec into instances.
Precedence: explicit argument, then `WIKITOOL_SEARCH_BACKEND`, then the Precedence: explicit argument, then `WIKITOOL_SEARCH_BACKEND`, then the
default. A comma-separated spec selects several and fuses their rankings. default. A comma-separated spec selects several and fuses their rankings.
`kb_dir`/`root` are handed to the backend rather than left to its own
defaults. Without them a caller could pass a corpus to `run_search` and
still have the backend walk `config.KB_DIR` - the query answered from one
tree and the pages read from another, with nothing saying so.
""" """
raw = spec or os.environ.get(ENV_VAR) or DEFAULT_BACKEND raw = spec or os.environ.get(ENV_VAR) or DEFAULT_BACKEND
names = [n.strip() for n in raw.split(",") if n.strip()] names = [n.strip() for n in raw.split(",") if n.strip()]
@@ -38,4 +49,4 @@ def resolve(spec: str | None = None) -> list[SearchBackend]:
f"unknown search backend(s): {', '.join(unknown)}. " f"unknown search backend(s): {', '.join(unknown)}. "
f"Available: {', '.join(sorted(BACKENDS))}" f"Available: {', '.join(sorted(BACKENDS))}"
) )
return [BACKENDS[name]() for name in names] return [BACKENDS[name](kb_dir, root) for name in names]
+49 -11
View File
@@ -4,7 +4,7 @@ Why shell out instead of scanning in Python: `rg` is already the retrieval
layer the agent instructions point at, it handles large trees fast, and its layer the agent instructions point at, it handles large trees fast, and its
`--json` mode gives line numbers and matched text without reparsing files. `--json` mode gives line numbers and matched text without reparsing files.
Two safety properties are load-bearing and must survive any edit here: Three safety properties are load-bearing and must survive any edit here:
1. The query is passed as an *argv element*, never through a shell. There is 1. The query is passed as an *argv element*, never through a shell. There is
no `shell=True` anywhere in this module, so a query containing `;`, `$(...)` no `shell=True` anywhere in this module, so a query containing `;`, `$(...)`
@@ -12,16 +12,19 @@ Two safety properties are load-bearing and must survive any edit here:
2. `--fixed-strings` is the default. A user-supplied regex is opt-in via 2. `--fixed-strings` is the default. A user-supplied regex is opt-in via
`--regex`, so an accidental `.*` in a search term is a literal, and a `--regex`, so an accidental `.*` in a search term is a literal, and a
pathological pattern cannot be introduced without asking for one. pathological pattern cannot be introduced without asking for one.
3. A user-supplied pattern is evaluated **only** by `rg`, whose engine is
linear in the input. Nothing here hands it to Python's `re`, which
backtracks - see `_contains`.
""" """
from __future__ import annotations from __future__ import annotations
import json import json
import re
import subprocess import subprocess
from pathlib import Path from pathlib import Path
from typing import Iterable from typing import Iterable
from chemenu import config from chemenu import config
from chemenu.errors import BackendError
from chemenu.page import Page from chemenu.page import Page
from chemenu.search.base import page_key from chemenu.search.base import page_key
from chemenu.search.filters import collection_of from chemenu.search.filters import collection_of
@@ -41,13 +44,22 @@ TITLE_WEIGHT = 5.0
SUMMARY_WEIGHT = 3.0 SUMMARY_WEIGHT = 3.0
LINE_WEIGHT = 1.0 LINE_WEIGHT = 1.0
# A hang-breaker, not a performance budget. A fixed-string search over this
# corpus costs 6 ms and a deliberately broad regex 1.4 s, so nothing legitimate
# comes near this; it exists so that a pathological pattern, a corpus on a
# stalled network mount, or an `rg` that never returns fails as an error
# instead of holding the caller open forever while its output buffers into the
# heap. The caller sees the ordinary `RipgrepFailed` path.
RIPGREP_TIMEOUT_SECONDS = 30.0
class RipgrepMissing(RuntimeError):
class RipgrepMissing(BackendError):
"""Raised when the `rg` executable is not on PATH.""" """Raised when the `rg` executable is not on PATH."""
class RipgrepFailed(RuntimeError): class RipgrepFailed(BackendError):
"""Raised when `rg` exits with an error (exit code 2 or above).""" """Raised when `rg` exits with an error (exit code 2 or above), or had to
be killed for running past `RIPGREP_TIMEOUT_SECONDS`."""
def build_argv(query: SearchQuery, root: Path) -> list[str]: def build_argv(query: SearchQuery, root: Path) -> list[str]:
@@ -99,7 +111,20 @@ class RipgrepBackend:
argv = build_argv(query, self.search_root) argv = build_argv(query, self.search_root)
try: try:
proc = subprocess.run(argv, capture_output=True, text=True, check=False) proc = subprocess.run(
argv,
capture_output=True,
text=True,
check=False,
timeout=RIPGREP_TIMEOUT_SECONDS,
)
except subprocess.TimeoutExpired as exc:
raise RipgrepFailed(
f"rg did not finish within {RIPGREP_TIMEOUT_SECONDS:g}s and was killed. "
"A search that takes this long is a pathological pattern or an "
"unresponsive corpus directory, not a slow answer - narrow the query "
"or use a fixed string instead of --regex."
) from exc
except FileNotFoundError as exc: # pragma: no cover - depends on host except FileNotFoundError as exc: # pragma: no cover - depends on host
raise RipgrepMissing( raise RipgrepMissing(
"ripgrep (rg) is not installed or not on PATH. It is the search " "ripgrep (rg) is not installed or not on PATH. It is the search "
@@ -141,13 +166,26 @@ class RipgrepBackend:
def _contains(haystack: str, query: SearchQuery) -> bool: def _contains(haystack: str, query: SearchQuery) -> bool:
"""Literal containment, used only for the title and summary ranking boosts.
It never evaluates `query.text` as a regex, even when `query.regex` is set.
It used to, via `re.search`, and Python's engine backtracks: `(\\w+\\s?)+$`
against 114 characters of ordinary page text does not terminate in eight
seconds, while a pattern that fails deterministically takes 0.2 ms - the
difference is the pattern, not the haystack. `build_hit` calls this twice
per hit, and a pattern as cheap as `\\w` matches every page, so one request
bought two unbounded searches per page in the corpus.
Deleting the branch rather than bounding it is the right trade: `rg` has
already applied the pattern with a linear engine by the time we get here,
and the page is a hit *because* of that. What is lost is only the extra
weight a regex hit in the title would have scored - and since a summary and
an H1 are themselves lines in the file, `rg` still counts them. A pattern
that is mostly literal (`longhorn`) still earns its boost through the test
below; one that is not gets ranked by match count alone.
"""
if not query.text: if not query.text:
return False return False
if query.regex:
try:
return re.search(query.text, haystack, re.IGNORECASE) is not None
except re.error:
return False
return query.text.lower() in haystack.lower() return query.text.lower() in haystack.lower()
+112
View File
@@ -0,0 +1,112 @@
"""The search core, with no CLI attached.
`run_search()` and the corpus loader used to live in `commands/search.py`,
which imports `typer` at module level and `rich` through `_util`. Any
in-process caller therefore dragged the whole CLI head in behind it - which
made the "two third-party packages" the read core actually needs (`yaml`,
`jsonschema`) an accounting fiction rather than a fact about the import graph.
Nothing here imports `typer`, `rich`, or anything under `chemenu.commands`.
That is the boundary, and it is worth keeping: `commands/search.py` is now the
adapter that turns these values into terminal output and these exceptions into
exit codes, and the MCP server (Gitea #19) is a second adapter over the same
functions rather than a second implementation of them.
Errors are raised, never printed: `PredicateError` for a bad `--field`,
`RipgrepMissing`/`RipgrepFailed` for the backend. All of them are
`ChemenuError` - see `chemenu/errors.py`.
"""
from __future__ import annotations
from pathlib import Path
from chemenu import config
from chemenu.frontmatter_io import read_page_with_error
from chemenu.kb_scan import iter_kb_pages
from chemenu.page import Page
from chemenu.search import filters
from chemenu.search.base import page_key
from chemenu.search.fuse import reciprocal_rank_fusion
from chemenu.search.ripgrep import build_hit
from chemenu.search.types import SearchHit, SearchQuery
def load_pages_by_path(kb_dir: Path | None = None, root: Path | None = None) -> dict[str, Page]:
"""Every page under `kb/`, keyed by repo-relative path.
Path-keyed rather than title-keyed on purpose: `load_kb_pages()` drops one
of two pages sharing a stem, and search should still find both - a
duplicate title is a lint finding, not a reason to hide a page.
"""
kb_dir = kb_dir or config.KB_DIR
root = root or config.ROOT
pages: dict[str, Page] = {}
for path in iter_kb_pages(kb_dir):
frontmatter, body, error = read_page_with_error(path)
pages[page_key(path, root)] = Page(
path=path, frontmatter=frontmatter, body=body, frontmatter_error=error
)
return pages
def unreadable_pages(pages: dict[str, Page]) -> list[dict[str, str]]:
"""The pages whose frontmatter could not be used, as `{path, reason}`.
Reported rather than swallowed. Such a page has no `confidence` and no
`kind`, so it silently drops out of every positive `--field` predicate -
including the low-confidence sweep that exists to find pages in exactly
that state. Saying nothing makes it look like a page that did not match;
an empty block is excluded, because a page can legitimately carry one.
"""
return [
{"path": key, "reason": page.frontmatter_error}
for key, page in sorted(pages.items())
if page.frontmatter_error and page.frontmatter_error != "empty frontmatter block"
]
def _sort_key(hit: SearchHit, field: str):
value = hit.as_dict().get(field)
if value is None:
# Missing values sort last in either direction rather than crashing on
# a None comparison.
return (1, "")
if isinstance(value, (int, float)):
return (0, value)
return (0, str(value).lower())
def sort_hits(hits: list[SearchHit], sort: str | None) -> list[SearchHit]:
"""Sort by a hit field. A leading `-` reverses, e.g. `--sort -confidence`."""
if not sort:
return hits
descending = sort.startswith("-")
field = sort.lstrip("-")
ordered = sorted(hits, key=lambda h: _sort_key(h, field), reverse=descending)
return ordered
def run_search(
query: SearchQuery,
pages: dict[str, Page],
backends: list,
kb_dir: Path | None = None,
) -> list[SearchHit]:
"""Answer a query. Pure: no I/O beyond whatever a backend does."""
filters.validate_fields(query.predicates, pages)
if query.text:
rankings = [backend.search(query, pages) for backend in backends]
hits = rankings[0] if len(rankings) == 1 else reciprocal_rank_fusion(rankings)
allowed = filters.apply_predicates(pages, query.predicates, kb_dir)
hits = [hit for hit in hits if hit.path in allowed]
else:
selected = filters.apply_predicates(pages, query.predicates, kb_dir)
hits = [
build_hit(page, key, [], query, backend="frontmatter", kb_dir=kb_dir)
for key, page in selected.items()
]
hits.sort(key=lambda h: h.title.lower())
hits = sort_hits(hits, query.sort)
return hits[: query.limit] if query.limit else hits
-47
View File
@@ -1,47 +0,0 @@
"""The section headings wikitool reads and writes inside a page body.
These headings are structural, not prose: `xref add` locates Relationships and
See Also by name, and `cite add` owns the trailing Footnotes block. An author
may add any other heading they like - only the ones named here are matched by
the tool, and only these have to stay predictable.
kb/CONTRACT.md's Language rule puts page prose in the KB language. That used to
force these three to stay English, because a translated heading did not error -
it made `xref add` append a *second* section, silently. This module removes that
constraint by making the vocabulary explicit in one place.
Each heading has one **canonical** name - what the tool writes - and any number
of **aliases** it still recognizes. That asymmetry is what lets a corpus migrate
page by page instead of all at once: a page still carrying `## Relationships` is
found and appended to correctly, and only takes the canonical name when the page
itself is translated. Removing an alias is therefore a breaking change for every
page not yet converted, not a cleanup.
"""
import re
RELATIONSHIPS = "Beziehungen"
SEE_ALSO = "Siehe auch"
FOOTNOTES = "Fußnoten"
ALIASES: dict[str, tuple[str, ...]] = {
RELATIONSHIPS: ("Relationships",),
SEE_ALSO: ("See Also",),
FOOTNOTES: ("Footnotes",),
}
def names(canonical: str) -> tuple[str, ...]:
"""Every name `canonical` is recognized under, canonical first."""
return (canonical, *ALIASES.get(canonical, ()))
def heading_re(canonical: str) -> re.Pattern[str]:
"""Match a `## <heading>` line for `canonical` or any of its aliases."""
alternation = "|".join(re.escape(name) for name in names(canonical))
return re.compile(rf"^## (?:{alternation})[ \t]*$", re.MULTILINE)
def is_known(heading: str) -> bool:
"""True if `heading` is a canonical name or an alias of one."""
return any(heading in names(canonical) for canonical in ALIASES)

Some files were not shown because too many files have changed in this diff Show More