Chemenu 2.1.0 - deterministischer Wissenskompiler
Chemenu kompiliert Rohnotizen zu einem verlinkten, quellengebundenen Wiki: raw/ -> types/ + tools/ -> kb/ -> reports/. Was mechanisch ist, macht tools/wikitool; was Urteil braucht, macht ein Agent unter Contracts, deren Grenzen in Code durchgesetzt sind statt im Prompt. Dieser Commit ist der Startpunkt der oeffentlichen Historie. Die vorherige Entwicklung fand in einer privaten Instanz statt und ist nicht Teil dieses Repositorys; ihre Erzaehlung steht vollstaendig in CHANGES.md, das mit 44 Eintraegen von 0.1.0 bis 2.1.0 erhalten geblieben ist. Der mitgelieferte Korpus ist ein Testbett und eine Demo: 170 Seiten ueber den Stack selbst - Gates, Lint, Versionierung, Suche, das Wiki-Muster. Er dokumentiert das Werkzeug mit den eigenen Mitteln des Werkzeugs. Lizenz: AGPL-3.0 fuer den Stack (tools/, types/), CC-BY-4.0 fuer die Inhalte. Die Grenze zwischen beiden ist der Dateiplan, den dist export berechnet - siehe NOTICE.
This commit is contained in:
+139
@@ -0,0 +1,139 @@
|
||||
# tools/
|
||||
|
||||
Developer documentation for `wikitool` - how the CLI is built, how to change it,
|
||||
and how to run its tests.
|
||||
|
||||
**This is not the command reference.** That is [CONTRACT.md](CONTRACT.md), which
|
||||
`wikitool docs verify` checks against the registered commands. Copying the
|
||||
command table here would create a second copy that drifts, so this file
|
||||
deliberately has none - and `docs verify` now enforces that.
|
||||
|
||||
| Document | Audience |
|
||||
|---|---|
|
||||
| `README.md` (this file) | Humans working *on* wikitool |
|
||||
| [`CONTRACT.md`](CONTRACT.md) | Agents working *with* wikitool - commands, error contracts, maintenance schedule |
|
||||
| [`../AGENTS.md`](../AGENTS.md) | The invariants that say when using a command is mandatory |
|
||||
| [`../CHANGES.md`](../CHANGES.md) | What changed in the stack, and when |
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
cd tools
|
||||
python3 -m venv .venv
|
||||
.venv/bin/pip install -r requirements.txt
|
||||
```
|
||||
|
||||
`jsonschema` and `PyYAML` are hard dependencies, not optional extras: schema
|
||||
validation is the tool's whole safety net, so `cli.py` fails loudly with the
|
||||
install command rather than degrading silently.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
tools/
|
||||
wikitool entry point
|
||||
chemenu/
|
||||
cli.py Typer app: registers every command, runs the budget gate
|
||||
config.py repo layout constants (ROOT, RAW_DIR, KB_DIR, WORK_DIR, ...)
|
||||
kb_scan.py page iteration/loading over kb/
|
||||
kb_collections.py collection discovery (a directory with COLLECTION.md)
|
||||
type_resolver.py type-spec loading and schema resolution
|
||||
sections.py the section headings the tool reads and writes in a page body
|
||||
markdown_code.py masks code spans/fences so a page may show wiki notation, not only use it
|
||||
version.py the stack version: VERSION, the release stamp, the compatibility rule
|
||||
kb_state.py the KB version (.wikitool-kb.json) and the migration chain
|
||||
corpus_diff.py invariant comparison of kb/ between two revisions
|
||||
search/ pluggable search backends (base protocol, ripgrep, filters, fuse)
|
||||
commands/ one module per command or command group
|
||||
tests/ pytest suite
|
||||
```
|
||||
|
||||
## Adding a command
|
||||
|
||||
1. Write the module under `chemenu/commands/`. A group is a `typer.Typer()`
|
||||
app; a single command is a plain function.
|
||||
2. Register it in `cli.py` (`app.add_typer(...)` or `app.command(...)`).
|
||||
3. Add a row to [CONTRACT.md](CONTRACT.md)'s command table **and** to its error
|
||||
contract table. `docs verify` fails in both directions - an undocumented
|
||||
command and a documented non-command are equally reported.
|
||||
4. Add tests. Pure logic belongs in a function separate from the Typer callback
|
||||
(see `mass_update_gate_message`, `derive_run_key`, `run_export`), so a test
|
||||
does not need a CLI runner - and a Typer callback called directly from a test
|
||||
receives `OptionInfo` objects, not values, for any argument the test omits.
|
||||
5. Raise the version: `wikitool version bump --minor --title "..."` for a new
|
||||
command (`--patch` for a fix, `--major` when existing content has to be
|
||||
migrated, which then also needs a document under `instructions/migrations/`).
|
||||
A new command reaches every future instance, and CI's version gate refuses a
|
||||
stack change that moved no version.
|
||||
|
||||
Every command is counted against the iteration budget unless it is listed in
|
||||
`run_budget.SKIP_COMMANDS` / `SKIP_COMMAND_PATHS`. Only read-only retrieval
|
||||
belongs there.
|
||||
|
||||
## Design notes
|
||||
|
||||
**Deterministic by default.** Anything an LLM would otherwise re-derive -
|
||||
frontmatter, index statistics, cross-reference bookkeeping, log formatting,
|
||||
confidence arithmetic - is computed here so it comes out the same every time.
|
||||
|
||||
**Gates are code, not prompts.** The Mass-Update Gate (`git_publish.py`) and the
|
||||
Iteration Budget Gate (`run_budget.py`) refuse in-process, because a
|
||||
prompt-level limit is one an agent can talk itself past. Exemption lists are
|
||||
constants, never flags.
|
||||
|
||||
**Clearance is an exit code, not an instruction.** `EXIT_NEEDS_CLEARANCE`
|
||||
(42, in `commands/_util.py`) is a third outcome beside success and validation
|
||||
error, meaning "a human has to see this output first". The command's own
|
||||
message carries the reason, the evidence and the exact `--confirm <token>`
|
||||
re-run line; the instruction layer deliberately holds none of it, because a
|
||||
procedure written down in advance is one an agent can complete alone. Whether
|
||||
a human *actually* saw it is not enforced here - that question is answered in
|
||||
the eval layer (`evals/trajectory.py`, `clearance-ended-the-turn`).
|
||||
|
||||
**Section names are a vocabulary, not literals.** `xref add` writes into Relationships and See
|
||||
Also, and `cite add` owns the trailing Footnotes block, so those three headings are structure the
|
||||
tool matches on. They are named once in `sections.py`, and each has one canonical spelling - what
|
||||
the tool writes - plus aliases it still recognizes. That asymmetry is what let the wiki be
|
||||
translated page by page instead of atomically: an untranslated `## Relationships` is still found
|
||||
and appended to. Dropping an alias is therefore a breaking change for any page not yet converted,
|
||||
not a cleanup. Renaming a heading is a migration's job; no other command may do it as a side
|
||||
effect (see `cite_block_heading` in `provenance.py`, which exists solely so `cite sync` stays a
|
||||
no-op on an untranslated page).
|
||||
|
||||
**Generated output is never committed.** `reports/`, `.agents/skills/` and
|
||||
`.claude/skills/` are build output; `docs verify` carries canaries in both
|
||||
directions so an ignore rule can neither swallow tracked content nor stop
|
||||
ignoring generated copies.
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
cd tools && .venv/bin/python -m pytest -q
|
||||
```
|
||||
|
||||
`pytest.ini` sets the import path. Tests use `tmp_path` fixtures and monkeypatch
|
||||
`config` paths rather than touching the real `kb/`.
|
||||
|
||||
Coverage is optional locally and measured on every CI run:
|
||||
|
||||
```bash
|
||||
.venv/bin/pip install pytest-cov # one time, CI-only dependency
|
||||
.venv/bin/python -m pytest -q --cov
|
||||
```
|
||||
|
||||
`pytest-cov` is deliberately not in `requirements.txt`: that file is what an *instance*
|
||||
needs at runtime and ships with `dist export`, and an instance does not measure this
|
||||
suite. Configuration is `.coveragerc` (coverage.py does not read `pytest.ini`), which
|
||||
measures `chemenu/` without `chemenu/tests/` and sets no threshold - see EVALS.md
|
||||
for the measured number and how to read it.
|
||||
|
||||
**Two exceptions, and they bite.** `test_types_cmd.py` and `test_index_build.py` resolve the real
|
||||
`types/` through the shared `resolver`, because what they assert *is* that behaviour comes from the
|
||||
type-specs rather than from a hardcoded map. Editing a type-spec can therefore fail tests that look
|
||||
unrelated to it - translating the wiki broke six that way. Anything those tests pin must be
|
||||
structural: `dir:` values and layout order are asserted exactly, display titles are read from the
|
||||
spec at assert time, never written out. The same rule applies to the template fence - anchor on a
|
||||
section name from `sections`, not on a literal.
|
||||
|
||||
Assert on prose in a type-spec or a contract only when the prose is the subject of the test.
|
||||
Otherwise it is a tripwire that fires on an edit nobody connected to the test.
|
||||
Reference in New Issue
Block a user