diff --git a/AGENTS.md b/AGENTS.md index ab0af87..1b6b5c2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -275,3 +275,10 @@ and `tools/README.md` are part of the change that introduced a stage, a command not follow-up work: nobody comes back for them, and a document that describes a repo which no longer exists is worse than none. The mechanical half - command tables, contracts, ignore canaries - is checked by `tools/wikitool docs verify`; the prose half is yours. + +`docs/` pages are held to a different clock than those three. A README goes stale on every new +flag; a `docs/` page goes stale only when the reasoning it wrote down stops holding - a gate +that stops living in code, an ownership line that moves, a boundary redrawn - which is rarer +and not tied to any one commit. Nothing checks this by construction: a page there carries no +normative sentence (see [File naming](#file-naming)), so there is no rule for `docs verify` to +check, only a rationale for a session to notice has gone stale and to update or retire. diff --git a/CHANGES.md b/CHANGES.md index 317c467..d818cc9 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -20,6 +20,31 @@ their date-only headings. --- +## 4.3.1 - 2026-09-03 - docs/ befuellt - Stack-Hintergrund fuer vier Themen, Pflegeklausel in AGENTS.md ergaenzt + +**Author:** Torben Nehmer + +Gitea #45: die von #38 angelegte, bis dahin leere `docs/` bekommt ihre ersten vier Seiten - frisch +geschrieben, nicht durch Umzug aus `kb/` befuellt, jede ohne normativen Satz und mit Verweis auf +das bindende Dokument statt einer Wiederholung seiner Regeln: + +- `docs/pipeline-rationale.md` - warum `raw -> types/tools -> kb -> reports` vier getrennte Stufen + sind und was "never re-derive, always compile" praktisch bedeutet +- `docs/why-gates-are-code.md` - warum Mass-Update-, Publish-Remote- und Iteration-Budget-Gate in + `tools/wikitool` statt in einer Instruktion stehen +- `docs/ownership-and-templates.md` - der Unterschied zwischen stack-eigenen, verbatim + ausgelieferten Dateien und instanz-eigenen `.template`-Dateien +- `docs/version-model.md` - warum Drop-in-Kompatibilitaet und Migrationsbedarf zwei unabhaengige + Fragen sind, illustriert an der 2.0.0-Fallstudie + +**AGENTS.md § Changelog:** neue Klausel zur Pflege von `docs/`, ergaenzt neben der bestehenden +Regel zu `README.md`/`EVALS.md`/`tools/README.md`. Eine `docs/`-Seite veraltet nicht wie ein +README bei jedem neuen Flag, sondern nur, wenn die aufgeschriebene Begruendung selbst nicht mehr +traegt - per Konstruktion ungeprueft, da die Seite keinen normativen Satz enthaelt, den +`docs verify` pruefen koennte. + +--- + ## 4.3.0 - 2026-09-03 - docs/ als ausgelieferter Hintergrund-Ort; Decision-Seiten bleiben in kb/, Decay-Skip fuer concept_type: decision **Author:** Torben Nehmer diff --git a/VERSION b/VERSION index 8089590..f77856a 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -4.3.0 +4.3.1 diff --git a/docs/ownership-and-templates.md b/docs/ownership-and-templates.md new file mode 100644 index 0000000..7f24a9f --- /dev/null +++ b/docs/ownership-and-templates.md @@ -0,0 +1,71 @@ +# Ownership and Templates + +Chemenu ships two kinds of files side by side, and at a glance they look the same: both are +plain markdown, both sit in the repo root or under `kb/`, both get read at session start. But a +stack upgrade treats them completely differently. Some - [AGENTS.md](../AGENTS.md), +[kb/CONTRACT.md](../kb/CONTRACT.md), the per-stage contracts - are identical in every instance +that runs this stack and can simply be overwritten by the next release. Others - `USER.md`, +`SOUL.md`, `kb/CONVENTIONS.md`, `ENVIRONMENT.md` - describe one particular instance, and +overwriting them would silently erase a choice someone made on purpose. + +## Two different kinds of truth + +The stack-owned files describe how the tool works. `kb/CONTRACT.md` opens by saying it holds +what `tools/wikitool` enforces or what follows mechanically from how it operates - see +[kb/CONTRACT.md](../kb/CONTRACT.md), lines 10-13. That kind of statement doesn't vary by +instance: the compiler behaves the same way regardless of who is running it, so the sentence +describing that behavior can be copied byte-for-byte into every checkout without becoming +wrong anywhere. + +The instance-owned files describe a choice: which language pages are written in, what tone the +agent takes, who the operator is, which git remote is authoritative, which MCP servers are +reachable. None of that follows from the tool's mechanics - two instances of the identical +stack can answer all of these differently and both be correct. [AGENTS.md § Personalization](../AGENTS.md#personalization) +frames the split the same way for `kb/CONTRACT.md` versus `kb/CONVENTIONS.md`: "the split is by +who may change the sentence, not by what it is about." A rule about page structure could in +principle have been written per-instance too, but then every instance answering "not German" to +setup would be hand-editing a file the stack also ships, and the next `dist export` merge would +hand the instance's own file back to it, discarding the customization. + +## Why silent overwrite is the failure being designed against + +A stack update is meant to be a routine, low-risk operation: pull the latest release, get +whatever fixes and features shipped since the last one. That only stays low-risk if the update +knows which files it's allowed to touch. If `USER.md` or `kb/CONVENTIONS.md` were treated the +same as `AGENTS.md` - shipped and periodically re-copied - an upgrade would quietly replace a +description of *this* operator, in *this* language, with whatever placeholder or default the +stack maintainers wrote. The damage wouldn't be loud: nothing crashes, the files still parse, +the agent just starts acting on the wrong premises until someone notices the voice or the +language changed. + +Keeping the boundary at the file level, rather than trying to merge changes within a shared +file, means an upgrade never has to guess which lines are "stack" and which are "instance" - +the file itself already answers that. + +## Why a `.template`, not just an absent file + +The mechanism for instance-owned content is a `.template` file the distribution ships instead +of the real one - `USER.md.template`, `SOUL.md.template`, `kb/CONVENTIONS.md.template`, +`ENVIRONMENT.md.template`. An alternative would have been to ship nothing at all and let a +brand-new instance start from a blank page. The template exists because a blank page doesn't +tell [instructions/setup-instance.md](../instructions/setup-instance.md) what shape the answer +should take, and it gives nothing for a validator to check afterward. + +A template carries a placeholder value - a sentinel - in the fields that need a real answer. +Setup interviews the operator and replaces the sentinel with what they actually said. That +gives `doctor` a mechanical way to tell "personalized" from "not yet": a file that still +contains the sentinel hasn't been through setup, regardless of whether the file exists. That's +also why `ENVIRONMENT.md` only warrants a WARN rather than a FAIL when absent - see +[AGENTS.md § Environment](../AGENTS.md#environment) - while a missing or unfilled +`USER.md`/`SOUL.md`/`kb/CONVENTIONS.md` is a harder failure: `ENVIRONMENT.md` describes one +checkout among possibly several and is gitignored for that reason, so its absence is a normal +state rather than a sign setup was skipped. + +## The consequence in practice + +Running a stack upgrade against an existing instance boils down to: overwrite the verbatim +files, leave the `.template`-sourced files alone. The verbatim files are safe to replace +wholesale because they were never instance-specific to begin with - identical content going +back in changes nothing an instance actually decided. The template-sourced files were filled in +once, by a person, for a reason, and nothing about a newer release of the stack's mechanics +gives it standing to override that. diff --git a/docs/pipeline-rationale.md b/docs/pipeline-rationale.md new file mode 100644 index 0000000..b0673d8 --- /dev/null +++ b/docs/pipeline-rationale.md @@ -0,0 +1,64 @@ +# Why the pipeline has four stages + +Chemenu could, in principle, be one directory: drop a file in, ask a question, get an answer +computed fresh each time. It isn't built that way. The pipeline in +[AGENTS.md](../AGENTS.md#routing) - `raw/` -> `[types/ + tools/]` -> `kb/` -> `reports/`, with +`work/` alongside rather than inside it - separates *material* from *meaning* from +*byproduct*, and each seam exists because collapsing it costs something specific. + +## Why raw material stays untouched + +[raw/CONTRACT.md](../raw/CONTRACT.md) keeps a source exactly as it arrived. The reasoning is +simple once stated: the moment someone "cleans up" or reformats a source on the way in, the +thing later claims get checked against is no longer the thing that was actually said. An +immutable `raw/` means a citation always resolves to the original, not to somebody's tidied +memory of it. It also draws a trust boundary in one place instead of scattering it - everything +past `raw/` can be treated as reviewed, because nothing upstream of it silently already was. + +## Why extraction happens once, through a schema + +[types/type-spec.md](../types/type-spec.md) is what stands between a raw file and a `kb/` page: +a type-spec defines what a conforming instance of a page looks like, and the compiler +(`tools/wikitool`) applies it. The alternative - every query re-reading and re-interpreting the +source on demand - would mean paying the cost of understanding the material every single time, +and getting a slightly different answer each time depending on how the question was phrased. +Extracting once, against a fixed schema, turns "re-read and re-guess" into "look up what was +already compiled." That is the "never re-derive, always compile" principle from +[AGENTS.md](../AGENTS.md): understanding a source is expensive and worth doing exactly once, +after which it becomes a cheap, stable lookup. + +## Why a `kb/` page has to stand on its own + +[kb/CONTRACT.md](../kb/CONTRACT.md) sets the bar for the compiled layer: a page should answer a +future question without sending the reader back to the source it came from. That's the payoff +of compiling in the first place - if every answer still bottomed out in "go re-read the raw +file," the `kb/` layer would just be a pointer with extra steps, and the cost of extraction +would have bought nothing. A page that stands alone is what makes the corpus fast and +consistent to query: the work of understanding is already sitting there, done. + +## Why `reports/` doesn't need to be maintained + +[reports/CONTRACT.md](../reports/CONTRACT.md) treats most of what lands in `reports/` - +lint output, telemetry traces - as disposable. The structural content of a lint report can be +recomputed from the tree at any commit, so keeping an old copy around would just be a second +version of something the tool can already answer on demand, and a second copy is exactly the +kind of thing that quietly goes stale. Treating it as derived output rather than a fourth thing +to maintain means there is nothing there to fall out of sync - regenerating it is cheaper than +reconciling it. The one part that genuinely can't be recomputed - the judgment a pass produced - +is carried out into `kb/` or `kb/log.md` before the report itself is discarded, which is the +distinction between what's recomputable and what isn't. + +## Where `work/` fits + +[work/CONTRACT.md](../work/CONTRACT.md) describes a workshop, not a fifth pipeline stage: a +place for the notes, extracts and open decisions of a task that spans more than one session, on +its way toward becoming a `kb/` page. It sits beside the raw -> kb -> reports flow rather than +inside it - closer in spirit to a desk than to a conveyor belt. + +## The shape this produces + +Four stages, each answering a different question: `raw/` - what was actually said; `types/` + +`tools/` - how to turn that into structured understanding; `kb/` - what is now known; +`reports/` - what a pass over the corpus noticed in passing. Keeping them separate is what lets +each one be trusted for what it is, instead of every layer having to double as all four at +once. diff --git a/docs/version-model.md b/docs/version-model.md new file mode 100644 index 0000000..93b1376 --- /dev/null +++ b/docs/version-model.md @@ -0,0 +1,84 @@ +# Why the stack version splits compatibility from migration + +A stack version number looks like it answers one question. It actually answers two, and the two +are independent of each other. + +## Two questions, not one + +The first question is whether the new version is a drop-in replacement for the old one - whether +an existing instance can install it, and can also go back, without anyone doing hand-work. That +is what a version number *is*: a promise. The second question is whether the existing corpus in +`kb/` needs to change shape to keep working under the new version. These sound like the same +question, because most of the time a change that breaks compatibility also happens to touch +content, and most of the time a change that leaves content untouched also happens to be +compatible. The correlation is real; it just is not a law. `instructions/dev/version-parts.md` +carries the actual test for telling them apart and the steps that follow from it - this page is +about why the split exists at all. + +## Why "kb/ untouched" is not proof of anything + +The tempting shortcut is: if no page in `kb/` had to change, the bump can't be that serious. This +is exactly backwards for a class of changes that live entirely outside the corpus - a renamed +release artefact, a Python import path, an environment variable, the URL an instance's own +updater points at. None of those touch a single page. All of them can strand an existing +instance just as thoroughly as a rewritten type-spec would. The corpus is the part of the stack +that looks at itself; the compatibility question is about everything an instance depends on to +keep functioning, most of which the corpus never sees. + +## Reading compatibility off the leftmost non-zero component + +Semantic versioning gives every component a job, but only one of them is where an existing +instance's tooling actually looks to decide "is this safe." On a `2.x` stack that is MAJOR; on a +still-pre-1.0 `0.x` stack, by the same convention, it's MINOR - the leftmost slot that isn't +pinned to zero is the one an automated updater treats as the compatibility boundary. Bump +anything to its left, or bump that slot itself, and the promise changes. Everything to the right +of it can move as freely as the project likes without touching that promise. This is why the +question "is it boundary-crossing" always resolves to one specific digit, not to a feeling about +how big the change is. + +## Downgrade is half the promise + +It's natural to test compatibility by only asking "does the upgrade work." The other half - +"can an instance that upgraded put the old version back and land where it started" - carries +equal weight, and it's the half that's easy to forget because forward motion is what everyone is +testing for anyway. A state file the old version can no longer parse, a generated index in a new +shape, a stamp file that got renamed: none of these have to break the upgrade to break the +downgrade. An instance that can go forward but not back has already lost the property a +compatible version number is supposed to guarantee. + +## A promise made to a machine, not only to a person + +A human reading a changelog can absorb "this technically isn't compatible but it's fine, just +update those two things by hand." An instance's own update mechanism cannot. It reads a version +number, decides whether to pull the new release, and has no channel for nuance - which is exactly +why the update path itself is one of the sharpest ways to cross the boundary invisibly: if the +new version moves where updates come from, the very channel that would have told an instance to +adjust is the channel that just broke. The version number isn't documentation aimed at a reader; +it's an input consumed by code that has no other way to ask. + +## The 2.0.0 story + +This isn't hypothetical for this stack. The rebranding that produced Chemenu renamed the repo, +the release artefact, and the Python package - and left every page in `kb/` untouched. The first +instinct was a MINOR bump, on the reasoning that nothing in the corpus needed migrating. That +reasoning was correct on its own terms and answered the wrong question. Three things broke +underneath it: every existing instance's `update_url` pointed at a repo path that no longer +existed and, because it's a machine-written file, couldn't be hand-repaired; the release artefact +name changed, breaking every download script and pin against it; and the import name changed, +breaking anything importing the package from outside the shipped tree. The corpus had nothing to +say about any of this, because none of it lived in the corpus. + +What caught the mistake was a person looking at the diff and asking whether it really was a +drop-in replacement, not a validator. No check in `docs verify` or anywhere else confirms that a +version part was chosen correctly - it only confirms that a boundary-crossing bump documents +what it breaks. The 2.0.0 entry in `CHANGES.md` carries the corrected reasoning in full, and the +version bump that shipped it was `--major --no-migration`: boundary-crossing and untouched +corpus, at the same time, which is precisely the combination the two-question split exists to +make visible. + +## Where the procedure lives + +The drop-in test, the catalogue of changes that cross the boundary with no page touched, and the +steps for a boundary-crossing bump - the `--breaking` line, the migration document or +`--no-migration` reason, talking to the user before bumping - are one procedure, kept at one +place: [instructions/dev/version-parts.md](../instructions/dev/version-parts.md). diff --git a/docs/why-gates-are-code.md b/docs/why-gates-are-code.md new file mode 100644 index 0000000..91f8fc5 --- /dev/null +++ b/docs/why-gates-are-code.md @@ -0,0 +1,57 @@ +# Why gates are code + +Chemenu has three hard limits - the Mass-Update Gate, the Publish-Remote Gate, and the +Iteration Budget Gate - and all three live inside `tools/wikitool`, not in a paragraph of +instructions an agent reads and follows. The rules themselves, and what to do when one trips, +are in [AGENTS.md § Gates](../AGENTS.md#gates) and [instructions/gates.md](../instructions/gates.md). +This page is only about the design choice underneath them: why code, and why these three +mechanisms in particular. + +## A suggestion an agent can talk itself past + +An instruction like "don't publish too much at once" or "don't loop forever" lives in the same +place as every other piece of guidance a session is holding - alongside the task, the user's +last message, and whatever context made the moment feel urgent. Under pressure, or with a +plausible-sounding reason ("this batch is different, it's mechanical"), that guidance can be +reasoned around without anyone deciding to break a rule. Nothing enforces it; it just competes +for attention with everything else in the context window, and sometimes loses. + +A check compiled into the tool doesn't have that problem, because it isn't part of the +conversation at all. It runs before the command dispatches, regardless of how convincing the +case for skipping it seemed a moment earlier. The difference isn't that code is smarter than a +well-written instruction - it's that code doesn't get talked into anything. + +## Why three different mechanisms, not one + +The three gates ask three different questions, and each one's shape follows from what kind of +question it is. + +The Mass-Update Gate asks *is this change too large to publish unreviewed* - a judgment that +varies changeset by changeset, so it clears with a `--confirm` token tied to the specific +output the user just read. Approval is scoped to that one publish. + +The Publish-Remote Gate asks something underneath that: *is this even the right repository*. +That's not a per-push judgment, it's a standing property of the checkout - true or false for +every publish that checkout will ever attempt, not just this one. A confirm token would let an +agent clear it once and then treat the answer as settled, which is exactly backwards for a +question whose answer shouldn't move at all mid-session. The only way past it is the user +editing `.wikitool-remotes.json` directly, outside the gate's own flow. + +The Iteration Budget Gate asks a third kind of question - not "is this instance correct" but +"has this session stopped making progress." That's read from the shape of the call history +itself (call count, repeated identical calls), not from anything about the content of any one +call. + +## Numbers that come from measurement, not intuition + +The iteration ceiling didn't start where it sits now. It used to run 15-25, borrowed from a +general rule of thumb, until four real ingest runs measured 24, 26, 29 and 30 calls apiece - +every one of them an ordinary workflow doing nothing wrong, and every one of them at or past +where the old ceiling would have refused it. A limit that the normal case keeps tripping stops +functioning as a limit; it becomes background noise a session learns to route `--override-budget` +around as a matter of course, and the whole point of a hard-coded check is that it isn't supposed +to feel routine. + +That's the deeper reason these numbers live in a tool rather than in prose: prose is read once +and remembered loosely, but a threshold enforced every call is tested by every call, and a +threshold that fails its own test gets noticed and re-measured rather than quietly ignored.