diff --git a/AGENTS.md b/AGENTS.md index d9b93d5..7ac7553 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,6 +13,7 @@ maintained permanently; anything mechanical is done by `tools/wikitool`, never b - [Bootstrap](#bootstrap) - [Invariants](#invariants) +- [Conversation language](#conversation-language) - [File naming](#file-naming) - [Personalization](#personalization) - [Environment](#environment) @@ -81,6 +82,13 @@ These hold regardless of which skill is active or which part of this file is in 8. **One rule, one place.** Every normative rule lives at exactly one location; everywhere else links to it. Writing a second copy is how the two start disagreeing. +## Conversation language + +Write every message to the user - progress notes and the closing report included - in the KB +language: `language:` in `kb/CONVENTIONS.md`'s frontmatter, read at session start. This holds +whatever language a file, a tool's output or a source is in; quoted commands, paths and field +names stay as they are. + ## File naming What a file is called says who it is for and how it is loaded. This is a rule, not a habit; @@ -111,9 +119,10 @@ documents. What it may not carry is the same content twice - a README that resta contract is a second copy that drifts. `docs verify` enforces the specific case that already happened once: no README may hold a copy of the `wikitool` command table. -**Two languages, and which is which.** Which one a line is written in follows from the *For* +**Two rules for the language of a file.** Which one a line is written in follows from the *For* column above - who reads it - and from nothing else: not from who owns the file, and not from -whether it ever leaves this checkout. +whether it ever leaves this checkout. What an agent says in the chat is a third matter, and +has its own section: [Conversation language](#conversation-language). 1. **The control plane is written in English** - this file, `CLAUDE.md`, every `CONTRACT.md`, everything under `instructions/`, and the type-specs for non-page artifacts. Quoted @@ -131,20 +140,20 @@ whether it ever leaves this checkout. moves this rule; [docs/explanation/language-boundaries.md](docs/explanation/language-boundaries.md) has the reasoning. -2. **An agent speaks the instance's KB language**, whatever this file is written in. The value - lives in `kb/CONVENTIONS.md`'s `language:` and nowhere else; an instruction that models a - sentence for the user writes it in English like the rest of the control plane, and the agent - says it in that language. -3. **The stack's human documentation is written in English** - `README.md`, `INSTALL.md` and + + An instruction that models a sentence for the user writes that model in English like the rest + of the control plane; the agent says it in the KB language ([Conversation + language](#conversation-language)). +2. **The stack's human documentation is written in English** - `README.md`, `INSTALL.md` and everything under `docs/` - whatever an instance's `language:` says. It is the same text in every instance, and the parts of it that are generated come from English sources, so there is no instance whose language it could follow. `CHANGES.md` is a log, not documentation, and is not covered. -Nothing checks any of the three mechanically - a stop-word scan would flag the quoted vocabulary -above and miss a translated paragraph that reads cleanly. They are held up by whoever writes the +Nothing checks either mechanically - a stop-word scan would flag the quoted vocabulary above +and miss a translated paragraph that reads cleanly. They are held up by whoever writes the text, which is why [instructions/CONTRACT.md](instructions/CONTRACT.md) § "Writing an -instruction" names the first two at the step where that happens. +instruction" names the first one at the step where that happens. **`docs/` carries no normative sentence.** It holds the stack's human documentation - tutorials, how-to guides and reference, and under `docs/explanation/` why the stack is built the way it is: diff --git a/CHANGES.md b/CHANGES.md index a1331cd..f353296 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -14,7 +14,7 @@ place only: `instructions/dev/version-parts.md` § The candidate model. --- -## 10.0.0-beta.8 - 2026-10-11 - raw fetch hinter einem Login: raw login speichert die Session des Nutzers als Secret, raw fetch holt damit; gerenderte Seiten über die optionale Browser-Komponente, externe Abrufer pro Host +## 10.0.0-beta.9 - 2026-10-11 - Gesprächssprache: AGENTS.md § Conversation language legt fest, dass der Agent im Chat die KB-Sprache spricht; instructions sync schreibt den Sprachwert als generierte Zeile in jedes veröffentlichte SKILL.md, instructions verify und doctor melden eine geänderte Sprache als Drift **Author:** Torben Nehmer @@ -32,6 +32,7 @@ place only: `instructions/dev/version-parts.md` § The candidate model. - Site-Handler der Instanz als JSON-Dateien in sites/, nach denselben Bausteinen wie die des Stacks - die nun ebenfalls Datendateien sind - raw fetch --scope: eine Seite und die Unterseiten, die ihr Site-Handler anbietet, als ein Bündel in incoming/ - raw fetch hinter einem Login: raw login speichert die Session des Nutzers als Secret, raw fetch holt damit; gerenderte Seiten über die optionale Browser-Komponente, externe Abrufer pro Host +- Gesprächssprache: AGENTS.md § Conversation language legt fest, dass der Agent im Chat die KB-Sprache spricht; instructions sync schreibt den Sprachwert als generierte Zeile in jedes veröffentlichte SKILL.md, instructions verify und doctor melden eine geänderte Sprache als Drift **Low impact** - Ketten von Paketen: stack-dev schreibt ## Kette, stack-close nennt das nächste Glied diff --git a/VERSION b/VERSION index 1e1e1e7..3869d08 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -10.0.0-beta.8 +10.0.0-beta.9 diff --git a/docs/explanation/language-boundaries.md b/docs/explanation/language-boundaries.md index d8c9090..06d3943 100644 --- a/docs/explanation/language-boundaries.md +++ b/docs/explanation/language-boundaries.md @@ -7,7 +7,9 @@ tells an agent what to do - [AGENTS.md](../../AGENTS.md), every `CONTRACT.md`, e is the stack's own documentation for humans, for a different reason (§ "The stack's human documentation"). -The rule itself is in [AGENTS.md § File naming](../../AGENTS.md#file-naming). This page holds the +The rules themselves are in [AGENTS.md § File naming](../../AGENTS.md#file-naming) (the language +of a file) and [AGENTS.md § Conversation language](../../AGENTS.md#conversation-language) (the +language of what an agent says). This page holds the part that is not a rule: why the line runs where it does, why the English half is not a setting, and which argument for it turned out to be wrong. @@ -18,6 +20,7 @@ and which argument for it turned out to be wrong. - [Why the control plane's language is English](#why-the-control-planes-language-is-english) - [Why it is not a parameter](#why-it-is-not-a-parameter) - [What the KB language still decides](#what-the-kb-language-still-decides) + - [Why a rule alone did not hold](#why-a-rule-alone-did-not-hold) - [Where the line runs around a page type](#where-the-line-runs-around-a-page-type) - [The stack's human documentation](#the-stacks-human-documentation) - [What would put this back on the table](#what-would-put-this-back-on-the-table) @@ -105,6 +108,31 @@ So an operator who reads no English gets German pages and German answers from an English instructions. The English is what the machinery is written in, not what it says back - with one exception, the stack's documentation for humans (§ "The stack's human documentation"). +### Why a rule alone did not hold + +For a while the conversation rule was a sentence about files' languages in `AGENTS.md`, and an +instance on German pages still saw an agent that wrote its pages in German and everything else in +English: progress notes between tool calls, the closing report. Pages were right because the +step that writes a page reads `kb/CONVENTIONS.md` first; the chat was wrong because nothing had +put the value in front of the agent by then. Three things stacked up: + +- **The rule sat in the wrong section.** A paragraph about the language of files does not read as + a rule about speaking, and it was phrased as a statement of fact rather than an instruction. + It now has a section of its own. +- **The value was one hop away.** `AGENTS.md` ships verbatim to every instance and so cannot + carry the instance's language; it names the file that does. A short ingest has written several + English messages before it opens that file. +- **The context is English.** Instructions, tool output and error text all are, and one short + message from the operator is outweighed by them - the more so for a weaker model. + +So the value is also compiled into the published copy of each skill, as a one-line block under +its frontmatter that `wikitool instructions sync` writes: the language tag and a pointer to the +rule, nothing else. The rule stays in `AGENTS.md` and the value in `kb/CONVENTIONS.md`; the block +holds neither as text of its own, only puts the one in front of the task that needs it. It is a +tag, not a language name, because a table of names would be the compiler's opinion about +languages, which `conventions.py` deliberately has none of. A session with no skill is covered by +the `AGENTS.md` section alone. + ## Where the line runs around a page type A page type's contract is where the two languages meet most closely, and it is worth knowing diff --git a/docs/reference/commands/types-instructions-and-docs.md b/docs/reference/commands/types-instructions-and-docs.md index d5f7694..457e7c7 100644 --- a/docs/reference/commands/types-instructions-and-docs.md +++ b/docs/reference/commands/types-instructions-and-docs.md @@ -138,6 +138,7 @@ tools/wikitool instructions sync **Behaviour** - Publishes every `instructions//SKILL.md` into `.agents/skills/` and `.claude/skills/` as **copies**, and deletes published skills whose source is gone. +- Each published `SKILL.md` is its source plus one generated line directly under the frontmatter: the `language:` of `kb/CONVENTIONS.md` and a pointer to `AGENTS.md` § Conversation language, so the agent has the value in context when the task begins. The frontmatter and every other file are copied unchanged. With no language declared, the `SKILL.md` is copied unchanged too, and the success message says so. - Both targets are gitignored, so a fresh clone runs this once. - Re-running repairs a drifted copy: the source always wins. - `--force` is required only to replace a target directory that is not a published skill at all (no `SKILL.md` in it). @@ -186,7 +187,7 @@ tools/wikitool instructions verify - Flat instructions validate against `types/instruction.schema.yaml`, and each `SKILL.md` carries the frontmatter its harness reads. - No `SKILL.md` carries a relative markdown link: `sync` copies it to a different depth than the source, so a `SKILL.md` references a target as a repo-root-relative plain path instead (`instructions/CONTRACT.md` § "A skill's outbound reference is a plain path, not a link"). -- Every published copy is byte-identical to its source. Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout. +- Every published copy is what `sync` would write from its source: byte-identical, except for the generated language line in `SKILL.md` that the current `language:` of `kb/CONVENTIONS.md` calls for. Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout. - No instruction is left that nothing references; one marked `manual: true` must instead not be linked from AGENTS.md, CLAUDE.md or a skill. - Nothing under `instructions/dev/` is referenced from outside it; a `` block is exempt (`instructions/CONTRACT.md`). - Read-only. @@ -204,7 +205,7 @@ tools/wikitool instructions verify - `0` - success - `1` - Nothing found under `instructions/` at all, or a malformed instruction or `SKILL.md` - `1` - A `SKILL.md` carries a relative markdown link -- `1` - A published copy drifted from its source +- `1` - A published copy drifted from its source - including a language line that belongs to an earlier `language:` (the message names it as such) - `1` - An instruction nothing references, or a `manual: true` one that IS linked from AGENTS.md, CLAUDE.md or a skill and so risks running implicitly - `1` - Something under `instructions/dev/` is referenced from outside it and outside a `dist:strip` block diff --git a/instructions/CONTRACT.md b/instructions/CONTRACT.md index 814f949..1897d2a 100644 --- a/instructions/CONTRACT.md +++ b/instructions/CONTRACT.md @@ -160,7 +160,19 @@ Both targets are generated and gitignored. A fresh clone therefore has no skills Copies, not symlinks: a symlink cannot go stale but is unreliable on Windows checkouts and does not survive being archived or copied. The price of a copy is drift, and drift is what -`tools/wikitool instructions verify` checks - byte for byte against the source. +`tools/wikitool instructions verify` checks - against what `sync` would write from the source, +which is the source byte for byte plus one generated line. + +That line is the instance's conversation language. When `kb/CONVENTIONS.md` declares a +`language:`, `sync` writes `> **Conversation language: \`de\`** - see \`AGENTS.md\` § +Conversation language ...` as the first paragraph under each published `SKILL.md`'s frontmatter, +so the value is in context when the task starts, not a file read away (the rule is +[AGENTS.md § Conversation language](../AGENTS.md#conversation-language); the line carries the +value and a pointer to it, not a second copy of the rule). Only `SKILL.md` gets it, in both +target directories, and its frontmatter and every other file of the skill stay byte-identical. With no +language declared the copy is the source unchanged. A skill's source never contains the line, and +changing `language:` makes `verify` and `doctor` report the copies as drifted, naming the line as +the reason; re-running `sync` repairs it. ## Writing an instruction @@ -191,12 +203,13 @@ Scaffold with `tools/wikitool new instruction --name ""`; the contract is `instructions/migrations/` belong to the release they shipped with and are not rewritten. The stack's own instructions are held to this by a test in the origin repository; what an instance writes for itself is its own decision. -- **Write it in English, and let the agent speak the instance's language.** Both rules, and the - line between prose and quoted vocabulary, are stated once in - [AGENTS.md § File naming](../AGENTS.md#file-naming). They are named here because this is the - step where they are obeyed or lost: nothing checks either mechanically, and an instruction - that models a sentence for the user is where the two are easiest to confuse - the model is - written in English, the saying of it follows `kb/CONVENTIONS.md`'s `language:`. +- **Write it in English, and let the agent speak the instance's language.** The first rule, and + the line between prose and quoted vocabulary, are stated once in + [AGENTS.md § File naming](../AGENTS.md#file-naming), the second in + [AGENTS.md § Conversation language](../AGENTS.md#conversation-language). They are named here + because this is the step where they are obeyed or lost: nothing checks either mechanically, and + an instruction that models a sentence for the user is where the two are easiest to confuse - + the model is written in English, the saying of it follows `kb/CONVENTIONS.md`'s `language:`. ### A skill's H1 is a name, not an imperative @@ -251,9 +264,9 @@ the next session to read before adding one. ### A skill's outbound reference is a plain path, not a link -`tools/wikitool instructions sync` copies each `SKILL.md` byte for byte into -`.agents/skills//` and `.claude/skills//` (§ Publishing, above) - a different depth -than the source, and without the sibling files a relative link might expect. A markdown link +`tools/wikitool instructions sync` copies each `SKILL.md` into +`.agents/skills//` and `.claude/skills//` (§ Publishing, above; the generated +language line it adds is no link) - a different depth than the source, and without the sibling files a relative link might expect. A markdown link correct at `instructions//SKILL.md` (`../session-setup.md`, `../../kb/CONTRACT.md`) resolves to a different, usually nonexistent, file once copied: the number of `../` segments that reaches a target from `instructions/` does not reach the same target from diff --git a/instructions/dev/stack-build/SKILL.md b/instructions/dev/stack-build/SKILL.md index 869941a..c47ade7 100644 --- a/instructions/dev/stack-build/SKILL.md +++ b/instructions/dev/stack-build/SKILL.md @@ -72,7 +72,7 @@ session, or a fresh one after `/clear` - assume the second, and work from the bo `instructions/dev/doc-pull-through.md` has the table of which document that is, per surface, and its step 3 for the one part that is not prose: a reference file whose headings moved needs `tools/wikitool docs toc --apply`, never a hand-written list. Prose you write here is English, - whatever language the session is held in - `AGENTS.md` § File naming has both language rules. + whatever language the session is held in - `AGENTS.md` § File naming has the rule for files, § Conversation language the one for what you say. 6. **Publish and wait for CI** per `instructions/dev/publish-and-ci.md`: the local checks, `tools/wikitool publish`, then the run on the published commit. @@ -94,7 +94,7 @@ session, or a fresh one after `/clear` - assume the second, and work from the bo session on Opus at high effort, which works from the body and the diff. That is why the three fixed points above are not optional. - Close with the fixed line, in the instance's KB language per `AGENTS.md` § File naming: + Close with the fixed line, in the instance's KB language per `AGENTS.md` § Conversation language: > #N is published and CI is green (run ). Next: `/stack-close` - here, or after `/clear`. diff --git a/instructions/dev/stack-close/SKILL.md b/instructions/dev/stack-close/SKILL.md index 8fb5026..b789162 100644 --- a/instructions/dev/stack-close/SKILL.md +++ b/instructions/dev/stack-close/SKILL.md @@ -115,7 +115,7 @@ always will. - **No `## Kette` section:** give no line, as before. Close the phase with exactly one of these fixed lines, in the instance's KB language per - `AGENTS.md` § File naming. A link is *ready* when it carries `kind/build` and neither + `AGENTS.md` § Conversation language. A link is *ready* when it carries `kind/build` and neither `status/incoming` nor `status/unconfirmed`: > #N is closed. Next in the chain "": #M - `/stack-build #M`, here or after `/clear`. diff --git a/instructions/dev/stack-dev/SKILL.md b/instructions/dev/stack-dev/SKILL.md index feb20c6..80771d6 100644 --- a/instructions/dev/stack-dev/SKILL.md +++ b/instructions/dev/stack-dev/SKILL.md @@ -66,7 +66,7 @@ are split that way. - **A short one** - an existing spec refreshed against the tree: continue in this session with `/stack-build #N`. - Close with the fixed line, in the instance's KB language per `AGENTS.md` § File naming: + Close with the fixed line, in the instance's KB language per `AGENTS.md` § Conversation language: > #N is ready. Next: `/stack-build #N` - here, or after `/clear`. diff --git a/kb/CONVENTIONS.md b/kb/CONVENTIONS.md index cb312f9..5dc9fd7 100644 --- a/kb/CONVENTIONS.md +++ b/kb/CONVENTIONS.md @@ -51,9 +51,10 @@ the control plane and stays English - the same prose/identifier cut [kb/CONTRACT.md](CONTRACT.md#language-and-identifiers) makes inside a page, applied one level up. `raw/` is untouched ([raw/CONTRACT.md](../raw/CONTRACT.md)). -Two things follow from that value rather than being decided here, both stated once in -[AGENTS.md § File naming](../AGENTS.md#file-naming): the control plane stays English whatever an -instance writes its pages in, and an agent *speaks* the language named above. +Two things follow from that value rather than being decided here: the control plane stays English +whatever an instance writes its pages in ([AGENTS.md § File naming](../AGENTS.md#file-naming)), +and an agent *speaks* the language named above +([AGENTS.md § Conversation language](../AGENTS.md#conversation-language)). The first of those is not a setting this file withholds - it is not a setting at all. `language:` above is the only language value in the tree, and what it binds is page text; a control-plane diff --git a/kb/CONVENTIONS.md.template b/kb/CONVENTIONS.md.template index d7ce6d5..2e224f6 100644 --- a/kb/CONVENTIONS.md.template +++ b/kb/CONVENTIONS.md.template @@ -51,9 +51,10 @@ page, applied one level up. Adopting this template into a non-English instance t translating those blocks, not the whole file. `raw/` is untouched ([raw/CONTRACT.md](../raw/CONTRACT.md)). -Two things follow from that value rather than being decided here, both stated once in -[AGENTS.md § File naming](../AGENTS.md#file-naming): the control plane stays English whatever an -instance writes its pages in, and an agent *speaks* the language named above. +Two things follow from that value rather than being decided here: the control plane stays English +whatever an instance writes its pages in ([AGENTS.md § File naming](../AGENTS.md#file-naming)), +and an agent *speaks* the language named above +([AGENTS.md § Conversation language](../AGENTS.md#conversation-language)). The first of those is not a setting this file withholds - it is not a setting at all. `language:` above is the only language value in the tree, and what it binds is page text; a control-plane diff --git a/tools/CONTRACT.md b/tools/CONTRACT.md index 01bf243..e7d890b 100644 --- a/tools/CONTRACT.md +++ b/tools/CONTRACT.md @@ -2542,6 +2542,7 @@ Publish every `instructions//SKILL.md` into the harness skill directories. **NOTES** - Publishes every `instructions//SKILL.md` into `.agents/skills/` and `.claude/skills/` as **copies**, and deletes published skills whose source is gone. +- Each published `SKILL.md` is its source plus one generated line directly under the frontmatter: the `language:` of `kb/CONVENTIONS.md` and a pointer to `AGENTS.md` § Conversation language, so the agent has the value in context when the task begins. The frontmatter and every other file are copied unchanged. With no language declared, the `SKILL.md` is copied unchanged too, and the success message says so. - Both targets are gitignored, so a fresh clone runs this once. - Re-running repairs a drifted copy: the source always wins. - `--force` is required only to replace a target directory that is not a published skill at all (no `SKILL.md` in it). @@ -2577,7 +2578,7 @@ Check the instruction layer. - 0 success - 1 Nothing found under `instructions/` at all, or a malformed instruction or `SKILL.md` - 1 A `SKILL.md` carries a relative markdown link -- 1 A published copy drifted from its source +- 1 A published copy drifted from its source - including a language line that belongs to an earlier `language:` (the message names it as such) - 1 An instruction nothing references, or a `manual: true` one that IS linked from AGENTS.md, CLAUDE.md or a skill and so risks running implicitly - 1 Something under `instructions/dev/` is referenced from outside it and outside a `dist:strip` block @@ -2585,7 +2586,7 @@ Check the instruction layer. - Nothing found under `instructions/` at all, or a malformed instruction or `SKILL.md` -> Fix the flagged file, then re-run - A `SKILL.md` carries a relative markdown link -> Rewrite it as a repo-root-relative plain path, then re-run -- A published copy drifted from its source -> Re-run `instructions sync` - the source under `instructions/` always wins +- A published copy drifted from its source - including a language line that belongs to an earlier `language:` (the message names it as such) -> Re-run `instructions sync` - the source under `instructions/` always wins - An instruction nothing references, or a `manual: true` one that IS linked from AGENTS.md, CLAUDE.md or a skill and so risks running implicitly -> Link it from where it is used, or drop the link to a manual one, then re-run - Something under `instructions/dev/` is referenced from outside it and outside a `dist:strip` block -> Remove the reference or wrap it in a `dist:strip` block, then re-run @@ -2597,7 +2598,7 @@ Check the instruction layer. - Flat instructions validate against `types/instruction.schema.yaml`, and each `SKILL.md` carries the frontmatter its harness reads. - No `SKILL.md` carries a relative markdown link: `sync` copies it to a different depth than the source, so a `SKILL.md` references a target as a repo-root-relative plain path instead (`instructions/CONTRACT.md` § "A skill's outbound reference is a plain path, not a link"). -- Every published copy is byte-identical to its source. Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout. +- Every published copy is what `sync` would write from its source: byte-identical, except for the generated language line in `SKILL.md` that the current `language:` of `kb/CONVENTIONS.md` calls for. Missing *every* copy is reported as "run sync", not as drift - that is a clean checkout. - No instruction is left that nothing references; one marked `manual: true` must instead not be linked from AGENTS.md, CLAUDE.md or a skill. - Nothing under `instructions/dev/` is referenced from outside it; a `` block is exempt (`instructions/CONTRACT.md`). - Read-only. diff --git a/tools/chemenu/commands/doctor.py b/tools/chemenu/commands/doctor.py index 5d98124..770dc8b 100644 --- a/tools/chemenu/commands/doctor.py +++ b/tools/chemenu/commands/doctor.py @@ -288,7 +288,8 @@ def check_skills() -> Check: ) if drifted: return Check( - "skills", "FAIL", f"{len(drifted)} published copy/copies drifted from source", + "skills", "FAIL", + f"{len(drifted)} published copy/copies drifted from source, first: {drifted[0]}", "Run `tools/wikitool instructions sync`", ) return Check("skills", "OK", f"{expected} published copy/copies match their source") diff --git a/tools/chemenu/commands/instructions_cmd.py b/tools/chemenu/commands/instructions_cmd.py index 2f2db66..85ff2f0 100644 --- a/tools/chemenu/commands/instructions_cmd.py +++ b/tools/chemenu/commands/instructions_cmd.py @@ -12,9 +12,17 @@ nothing else). This reverses an earlier design that used relative symlinks. The symlink argument was that a link cannot go stale; the counter-arguments that won are that symlinks are unreliable on Windows checkouts and do not survive being archived or copied, and that this repo is meant to stay reproducible -elsewhere. The price of a copy is drift, so `verify` checks every copy byte for -byte against its source - the copy is derived truth, and derived truth is either -checked or absent. +elsewhere. The price of a copy is drift, so `verify` checks every copy against +what `sync` would write from its source - the copy is derived truth, and derived +truth is either checked or absent. + +The copy is byte-identical to its source except for one generated block: when +`kb/CONVENTIONS.md` declares a `language:`, `sync` inserts a line carrying that +tag directly under each `SKILL.md`'s frontmatter, so the value that decides what +language the agent speaks is in context when the task begins instead of one file +read away (AGENTS.md § Conversation language holds the rule; the block holds +only the value and a pointer to it). `render_skill_md` is the one place that text +is made, and both `sync` and `drift` use it. Both target directories are gitignored. A fresh clone has no skills until `sync` runs; `instructions/bootstrap.md` is the procedure, and `verify` says so rather @@ -44,7 +52,7 @@ from pathlib import Path import typer import yaml -from chemenu import cli_contract, config, markdown_code +from chemenu import cli_contract, config, conventions, markdown_code from chemenu.commands import dist_cmd, docs_verify from chemenu.commands._util import fail, rel_path, success from chemenu.type_resolver import resolver @@ -142,7 +150,66 @@ def _read_frontmatter(path: Path) -> tuple[dict | None, str]: return frontmatter, "" -def _copy_skill(source: Path, target: Path, force: bool) -> None: +# The closing line of the frontmatter, matched line-wise: `_read_frontmatter`'s +# `find("---")` would stop at a `---` inside a value. +_FRONTMATTER_RE = re.compile(r"\A---[ \t]*\r?\n.*?\r?\n---[ \t]*(\r?\n|\Z)", re.DOTALL) +_LANGUAGE_BLOCK_TAG_RE = re.compile(r"^> \*\*Conversation language: `([^`\n]+)`\*\*", re.MULTILINE) + + +def language_block(language: str) -> str: + """The generated line `sync` writes under a published skill's frontmatter. + + A tag, never a language name: a name table would be the compiler's opinion + about languages, which `conventions.py` deliberately has none of.""" + return ( + f"> **Conversation language: `{language}`** - see `AGENTS.md` § Conversation language " + "(value from `kb/CONVENTIONS.md` `language:`, written by `wikitool instructions sync`)." + ) + + +def render_skill_md(source_text: str, language: str | None) -> str: + """What the published `SKILL.md` holds for this source and language: the + source itself when no language is declared (or it has no closed + frontmatter to insert under), else the source with `language_block` as the + first paragraph after the frontmatter.""" + if not language: + return source_text + match = _FRONTMATTER_RE.match(source_text) + if match is None: + return source_text + end = match.end() + eol = "\r\n" if match.group(0).endswith("\r\n") else "\n" + head = source_text[:end] if match.group(1) else source_text + eol + return f"{head}{eol}{language_block(language)}{eol}{source_text[end:]}" + + +def _write_skill_md(target: Path, language: str | None) -> None: + path = target / SKILL_FILE + source_text = path.read_bytes().decode("utf-8") + rendered = render_skill_md(source_text, language) + if rendered != source_text: + path.write_bytes(rendered.encode("utf-8")) + + +def _skill_md_drift(source: Path, target: Path, language: str | None) -> str | None: + """How a published `SKILL.md` differs from what `sync` would write, or None. + + A difference that is only the language block says so: `language:` changed + since the last `sync`, which is not a hand edit and should not read as one.""" + source_text = (source / SKILL_FILE).read_bytes().decode("utf-8") + target_text = (target / SKILL_FILE).read_bytes().decode("utf-8") + if target_text == render_skill_md(source_text, language): + return None + found = _LANGUAGE_BLOCK_TAG_RE.search(target_text) + stated = f"conventions say `{language}`" if language else "conventions declare no language" + if found and target_text == render_skill_md(source_text, found.group(1)): + return f"language block is for `{found.group(1)}`, {stated}" + if language and target_text == source_text: + return f"language block missing, {stated}" + return f"differs in {SKILL_FILE}" + + +def _copy_skill(source: Path, target: Path, force: bool, language: str | None) -> None: if target.is_symlink(): # Left over from the symlink-based mirror this replaced. target.unlink() @@ -155,6 +222,7 @@ def _copy_skill(source: Path, target: Path, force: bool) -> None: ) shutil.rmtree(target) shutil.copytree(source, target) + _write_skill_md(target, language) def _looks_like_a_published_skill(target: Path) -> bool: @@ -176,13 +244,16 @@ def drift(source: Path, target: Path) -> str | None: if target.is_symlink(): return "is a symlink, not a copy" comparison = filecmp.dircmp(str(source), str(target)) - if comparison.diff_files: - return f"differs in {', '.join(sorted(comparison.diff_files))}" + # SKILL.md is the one file that is not a byte copy; it is judged against its + # rendering below. + differing = sorted(set(comparison.diff_files) - {SKILL_FILE}) + if differing: + return f"differs in {', '.join(differing)}" if comparison.left_only: return f"missing {', '.join(sorted(comparison.left_only))}" if comparison.right_only: return f"has extra {', '.join(sorted(comparison.right_only))}" - return None + return _skill_md_drift(source, target, conventions.language()) def _reference_haystacks( @@ -381,6 +452,11 @@ def check_skill_reference_paths() -> list[str]: notes=( "Publishes every `instructions//SKILL.md` into `.agents/skills/` and " "`.claude/skills/` as **copies**, and deletes published skills whose source is gone.", + "Each published `SKILL.md` is its source plus one generated line directly under the " + "frontmatter: the `language:` of `kb/CONVENTIONS.md` and a pointer to `AGENTS.md` § " + "Conversation language, so the agent has the value in context when the task begins. The " + "frontmatter and every other file are copied unchanged. With no language declared, the " + "`SKILL.md` is copied unchanged too, and the success message says so.", "Both targets are gitignored, so a fresh clone runs this once.", "Re-running repairs a drifted copy: the source always wins.", "`--force` is required only to replace a target directory that is not a published " @@ -426,12 +502,13 @@ def sync( fail(f"No skills found under {rel_path(config.INSTRUCTIONS_DIR)}.") names = {source.name for source in sources} + language = conventions.language() published = [] removed = [] for target_root in target_dirs(): target_root.mkdir(parents=True, exist_ok=True) for source in sources: - _copy_skill(source, target_root / source.name, force) + _copy_skill(source, target_root / source.name, force, language) # A skill that no longer exists must not keep being offered. for stale in sorted(target_root.iterdir()): if stale.name not in names: @@ -443,7 +520,8 @@ def sync( published.append(rel_path(target_root)) suffix = f", removed {len(removed)} stale" if removed else "" - success(f"Published {len(sources)} skill(s) to {' and '.join(published)}{suffix}") + stamped = f"conversation language `{language}`" if language else "no language declared" + success(f"Published {len(sources)} skill(s) to {' and '.join(published)} ({stamped}){suffix}") @app.command("verify") @@ -464,8 +542,10 @@ def sync( "than the source, so a `SKILL.md` references a target as a repo-root-relative plain " "path instead (`instructions/CONTRACT.md` § \"A skill's outbound reference is a plain " "path, not a link\").", - "Every published copy is byte-identical to its source. Missing *every* copy is " - "reported as \"run sync\", not as drift - that is a clean checkout.", + "Every published copy is what `sync` would write from its source: byte-identical, except " + "for the generated language line in `SKILL.md` that the current `language:` of " + "`kb/CONVENTIONS.md` calls for. Missing *every* copy is reported as \"run sync\", not as " + "drift - that is a clean checkout.", "No instruction is left that nothing references; one marked `manual: true` must " "instead not be linked from AGENTS.md, CLAUDE.md or a skill.", "Nothing under `instructions/dev/` is referenced from outside it; a " @@ -483,7 +563,8 @@ def sync( reaction="Rewrite it as a repo-root-relative plain path, then re-run", ), cli_contract.Failure( - cause="A published copy drifted from its source", + cause="A published copy drifted from its source - including a language line that " + "belongs to an earlier `language:` (the message names it as such)", reaction="Re-run `instructions sync` - the source under `instructions/` always wins", ), cli_contract.Failure( @@ -510,7 +591,7 @@ def sync( ), )) def verify(): - """Check instructions/ against its type, that no skill carries a relative markdown link, and every published copy against its source.""" + """Check instructions/ against its type, that no skill carries a relative markdown link, and every published copy against what sync would write from its source.""" sources = skill_dirs() instructions = instruction_files() if not sources and not instructions: diff --git a/tools/chemenu/tests/test_doctor.py b/tools/chemenu/tests/test_doctor.py index 9773c96..0696939 100644 --- a/tools/chemenu/tests/test_doctor.py +++ b/tools/chemenu/tests/test_doctor.py @@ -461,6 +461,22 @@ def test_unpublished_skills_fail(instance): assert _status(checks, "skills") == "FAIL" +def test_a_changed_conversation_language_fails_skills_and_names_the_block(instance): + (config.KB_DIR / conventions.CONVENTIONS_FILENAME).write_text( + "---\nlanguage: de\nprofile: none\nsections:\n links: Relationships\n" + " footnotes: Footnotes\n---\n\n# conventions\n", + encoding="utf-8", + ) + conventions.reset_cache() + checks = doctor.run_doctor() + assert _status(checks, "skills") == "FAIL" + assert "2 published copy/copies drifted" in _detail(checks, "skills") + assert "language block is for `en`, conventions say `de`" in _detail(checks, "skills") + + instructions_cmd.sync(force=False) + assert _status(doctor.run_doctor(), "skills") == "OK" + + def test_no_author_fails(instance, monkeypatch): monkeypatch.delenv("WIKI_AUTHOR", raising=False) monkeypatch.setattr(config, "default_author", lambda: None) diff --git a/tools/chemenu/tests/test_instructions_cmd.py b/tools/chemenu/tests/test_instructions_cmd.py index 433f25e..b84143a 100644 --- a/tools/chemenu/tests/test_instructions_cmd.py +++ b/tools/chemenu/tests/test_instructions_cmd.py @@ -10,7 +10,7 @@ from pathlib import Path import pytest import typer -from chemenu import config +from chemenu import config, conventions from chemenu.tests.conftest import use_shipped_type_specs from chemenu.commands import instructions_cmd @@ -230,6 +230,184 @@ def test_drift_detects_missing_edited_and_extra(layer): assert "extra" in instructions_cmd.drift(source, target) +# --- conversation language block -------------------------------------------- + + +def _declare_language(root: Path, tag: str | None) -> None: + path = root / "kb" / "CONVENTIONS.md" + if tag is None: + path.unlink(missing_ok=True) + else: + path.write_text(f"---\nlanguage: {tag}\nprofile: none\n---\n\n# conventions\n", encoding="utf-8") + conventions.reset_cache() + + +def _rich_skill(layer) -> Path: + """A skill whose frontmatter has a flag and a value holding `---`, plus a sibling file.""" + source = layer / "instructions" / "wiki-demo" + (source / "SKILL.md").write_text( + "---\nname: wiki-demo\ndescription: Demo --- skill.\ndisable-model-invocation: true\n---\n\n# Demo\n\nSee gates.md.\n", + encoding="utf-8", + ) + (source / "reference.md").write_text("# reference\n", encoding="utf-8") + return source + + +def _block_lines(text: str) -> list[str]: + return [line for line in text.splitlines() if line.startswith("> **Conversation language:")] + + +def test_sync_writes_the_language_block_directly_under_the_frontmatter_in_both_targets(layer): + source = _rich_skill(layer) + _declare_language(layer, "de") + instructions_cmd.sync(force=False) + + source_text = (source / "SKILL.md").read_text(encoding="utf-8") + frontmatter_end = source_text.index("---\n\n# Demo") + len("---\n") + for harness in (".agents", ".claude"): + copy = _skill_copy(layer, harness) + text = (copy / "SKILL.md").read_text(encoding="utf-8") + assert len(_block_lines(text)) == 1 + assert text[:frontmatter_end] == source_text[:frontmatter_end] + after = text[frontmatter_end:].split("\n") + assert after[0] == "" and after[1].startswith("> **Conversation language: `de`**") + assert "`AGENTS.md` § Conversation language" in after[1] + assert text.replace(after[1] + "\n\n", "", 1) == source_text + assert (copy / "reference.md").read_bytes() == (source / "reference.md").read_bytes() + assert instructions_cmd.drift(source, _skill_copy(layer, ".agents")) is None + assert instructions_cmd.drift(source, _skill_copy(layer, ".claude")) is None + + +def test_the_block_is_the_tag_and_never_a_language_name(layer): + _declare_language(layer, "pt-BR") + instructions_cmd.sync(force=False) + text = (_skill_copy(layer, ".claude") / "SKILL.md").read_text(encoding="utf-8") + assert "`pt-BR`" in _block_lines(text)[0] + assert "Portuguese" not in text + + +def test_a_second_sync_is_byte_identical_and_adds_no_second_block(layer): + _declare_language(layer, "de") + instructions_cmd.sync(force=False) + first = (_skill_copy(layer, ".agents") / "SKILL.md").read_bytes() + instructions_cmd.sync(force=False) + second = (_skill_copy(layer, ".agents") / "SKILL.md").read_bytes() + assert first == second + assert len(_block_lines(second.decode("utf-8"))) == 1 + + +def test_without_a_declared_language_every_copy_is_the_source(layer, capsys): + source = _rich_skill(layer) + _declare_language(layer, None) + instructions_cmd.sync(force=False) + assert "no language declared" in " ".join(capsys.readouterr().out.split()) + for harness in (".agents", ".claude"): + copy = _skill_copy(layer, harness) + assert (copy / "SKILL.md").read_bytes() == (source / "SKILL.md").read_bytes() + assert instructions_cmd.drift(source, copy) is None + + +def test_sync_names_the_language_it_stamped(layer, capsys): + _declare_language(layer, "de") + instructions_cmd.sync(force=False) + assert "conversation language `de`" in " ".join(capsys.readouterr().out.split()) + + +def test_a_skill_without_closed_frontmatter_is_copied_unchanged(layer): + source = layer / "instructions" / "wiki-demo" + (source / "SKILL.md").write_text("---\nname: wiki-demo\n# never closed\n", encoding="utf-8") + _declare_language(layer, "de") + instructions_cmd.sync(force=False) + assert (_skill_copy(layer, ".claude") / "SKILL.md").read_bytes() == (source / "SKILL.md").read_bytes() + + +def test_a_crlf_skill_keeps_its_line_endings(layer): + source = layer / "instructions" / "wiki-demo" + (source / "SKILL.md").write_bytes(b"---\r\nname: wiki-demo\r\ndescription: Demo.\r\n---\r\n\r\n# Demo\r\n") + _declare_language(layer, "de") + instructions_cmd.sync(force=False) + data = (_skill_copy(layer, ".claude") / "SKILL.md").read_bytes() + assert data.count(b"\n") == data.count(b"\r\n") + assert instructions_cmd.drift(source, _skill_copy(layer, ".claude")) is None + + +def test_changing_the_language_after_sync_is_drift_that_names_the_block(layer): + source = layer / "instructions" / "wiki-demo" + _declare_language(layer, "en") + instructions_cmd.sync(force=False) + _declare_language(layer, "de") + for harness in (".agents", ".claude"): + assert instructions_cmd.drift(source, _skill_copy(layer, harness)) == ( + "language block is for `en`, conventions say `de`" + ) + instructions_cmd.sync(force=False) + assert instructions_cmd.drift(source, _skill_copy(layer, ".agents")) is None + + +def test_declaring_a_language_after_a_plain_sync_is_drift_that_names_the_block(layer): + source = layer / "instructions" / "wiki-demo" + instructions_cmd.sync(force=False) + _declare_language(layer, "de") + assert instructions_cmd.drift(source, _skill_copy(layer, ".agents")) == ( + "language block missing, conventions say `de`" + ) + + +def test_dropping_the_language_after_sync_is_drift_that_names_the_block(layer): + source = layer / "instructions" / "wiki-demo" + _declare_language(layer, "de") + instructions_cmd.sync(force=False) + _declare_language(layer, None) + assert instructions_cmd.drift(source, _skill_copy(layer, ".agents")) == ( + "language block is for `de`, conventions declare no language" + ) + + +def test_a_hand_edit_next_to_the_block_is_still_reported_as_a_difference(layer): + source = layer / "instructions" / "wiki-demo" + _declare_language(layer, "de") + instructions_cmd.sync(force=False) + target = _skill_copy(layer, ".agents") / "SKILL.md" + target.write_text(target.read_text(encoding="utf-8") + "extra\n", encoding="utf-8") + assert instructions_cmd.drift(source, _skill_copy(layer, ".agents")) == "differs in SKILL.md" + + +def test_a_hand_edit_of_another_file_is_still_reported(layer): + source = _rich_skill(layer) + _declare_language(layer, "de") + instructions_cmd.sync(force=False) + (_skill_copy(layer, ".claude") / "reference.md").write_text("# edited\n", encoding="utf-8") + assert instructions_cmd.drift(source, _skill_copy(layer, ".claude")) == "differs in reference.md" + + +def test_verify_reports_a_language_change_per_copy_and_sync_repairs_it(layer, capsys): + _declare_language(layer, "en") + instructions_cmd.sync(force=False) + instructions_cmd.verify() + capsys.readouterr() + + _declare_language(layer, "de") + with pytest.raises(typer.Exit): + instructions_cmd.verify() + out = capsys.readouterr().out + for harness in (".agents", ".claude"): + assert f"{harness}/skills/wiki-demo: language block is for `en`, conventions say `de`" in out + + instructions_cmd.sync(force=False) + instructions_cmd.verify() + assert "match their source" in " ".join(capsys.readouterr().out.split()) + + +def test_every_real_skill_takes_the_block(): + """A source whose frontmatter the insertion cannot find would be published + without the language and nothing would say so.""" + for source in instructions_cmd.skill_dirs(): + text = (source / "SKILL.md").read_text(encoding="utf-8") + rendered = instructions_cmd.render_skill_md(text, "de") + assert rendered != text, source.name + assert len(_block_lines(rendered)) == 1, source.name + + # --- verify -----------------------------------------------------------------