--- type: types/instruction.md name: migrate-corpus description: Change the shape of every kb/ page at once - a schema field, a vocabulary, a language - in planned units, with a mechanical check per unit and a recorded KB version at the end. --- # Migrate the corpus A change that touches the *shape* of pages rather than their content: a new required frontmatter field, a renamed enum value, a section heading vocabulary, a language. It is not an ingest and not a lint fix - nothing new is learned, the same knowledge is restated in a new form. The failure mode is therefore specific and quiet: **something present before is missing afterwards**, and the corpus is still internally consistent, so `lint` reports nothing. Every rule below was paid for once already. The German translation of 248 pages found four defects this way - a dropped citation that silently unsourced a claim, a dropped wikilink, an invented one, and a translated H1 - and three of the four had unchanged link *sets* and only changed counts. ## Contents - [When to run](#when-to-run) - [Steps](#steps) - [Decision points](#decision-points) - [Writing the migration document](#writing-the-migration-document) - [Scope](#scope) ## When to run A change that would otherwise be applied to more than a handful of pages by hand, or any change declared by a migration document under `instructions/migrations/`. A single page is `wiki-manage`; a raw tree is [ingest-large-tree.md](ingest-large-tree.md). ## Steps 1. **Open a workshop.** `tools/wikitool work new --key ` - not `ingest-`, which is reserved for keys derived from a `raw/` path. `plan.md` cuts the corpus into units and says why each cut falls where it does; `README.md` carries the closing condition and the decisions made so far. See [work/CONTRACT.md](../work/CONTRACT.md). 2. **Size the units by the iteration budget, not by feel.** One unit costs roughly `N × touch` + `index rebuild` + `log append` + `publish` (twice - the Mass-Update Gate refuses once and publishes on the confirm), plus `sources rebuild-index` if it contains source pages. Against the 60-call ceiling that puts the ceiling near 55 pages; aim for 48 or fewer. **Units and publishes are not the same boundary.** The budget is per session id; the gate is per publish. Several units may run back to back, each with its own `WIKITOOL_SESSION_ID="/u"`, and publish once together - which is what the written unit boundaries in `plan.md` make legitimate rather than a way around a gate refusal (see [gates.md](gates.md)). 3. **Rewrite the unit's pages.** Bodies only. Frontmatter is written with `touch`, never by hand, and never by a subagent. 4. **Check mechanically, before anything else:** ```bash tools/wikitool migrate verify --from HEAD --path kb/ --fail-on-error ``` This is the step the whole procedure exists for. It compares wikilink and citation **counts**, footnote definitions, H1 and structural frontmatter against the last commit. Run it before the summaries, before `lint`, before anything - it is the cheapest place to catch a subagent that helpfully translated a link target. 5. **Write the summaries yourself** with `touch --summary`, from the original. Never paste a subagent's proposal unread: they embellish, and a summary is a claim about the page. 6. **`index rebuild`, then `lint` - and read the whole report**, not only the sections this unit could plausibly have touched. The translation's first unit had a frontmatter round-trip bug that surfaced as a schema error on a field nobody had edited. 7. **`log append`, then publish** through [publish-cycle.md](publish-cycle.md). Expect exit 42 on a corpus-sized change; reproduce the breakdown for the user and wait. 8. **Carry the vocabulary between units.** Terminology settled in unit 5 and re-decided in unit 9 is the failure a glossary file in the workshop exists to prevent. Add to it *before* dispatching the next unit. 9. **Record the migration** once the last unit is published: ```bash tools/wikitool migrate done --pages ``` This advances `kb_version` in `.wikitool-kb.json`. It refuses any version that is not the next link in the chain, so a multi-step upgrade cannot silently skip one. 10. **Close the workshop** per [work/CONTRACT.md](../work/CONTRACT.md), after promoting whatever outlives the run. The translation's glossary became [german-terminology.md](german-terminology.md); its checklist and unit plan died with the directory, correctly. ## Decision points - **Can the change be made backwards-compatible instead?** Prefer it. A vocabulary migration does not need a flag day: `tools/chemenu/sections.py` gives each heading one canonical name and any number of aliases, so a page is found under the old name and takes the new one only when it is rewritten. Removing an alias afterwards is a second breaking change, not a cleanup. - **Mechanical or assisted?** A rename with a fixed rule is `mechanical` and wants a script; a change needing a judgment call per page is `assisted` and wants this procedure. There is no `migrate run` today - `migration_kind` describes the work, it does not perform it. - **The check finds something mid-unit.** Fix it in that unit and re-run `verify`. Never carry a finding into the next unit "to fix later": the next unit's diff baseline is this unit's commit, so an uncorrected drop becomes invisible. - **Contradiction with an existing page.** Never overwrite. Record both, ask the user, and hedge the page's prose to the weaker of the two sources if it stays unresolved (kb/CONVENTIONS.md § Hedging). ## Writing the migration document A migration that a distributed instance must also run is a `manual: true` instruction under `instructions/migrations/-.md`, carrying `migrates_to:` and `migration_kind:`. `tools/wikitool migrate status` builds the outstanding chain from those files, and `version bump` refuses a compatibility-breaking release that has none. Write it for a reader who has the new machinery and the old content, and who is not you: what changed, which pages are affected, how to tell a migrated page from an unmigrated one, and what `migrate verify` should report when it is done. **Baseline: 1.0.0.** Migrations that predate it - the type-system move, the `confidence_base` backfill, the German section headings, the translation itself - have no documents and will not get any. An instance older than that is re-exported, not migrated. ## Scope For `kb/` content. A single page is `wiki-manage`; a `raw/` tree is [ingest-large-tree.md](ingest-large-tree.md). Changing the machinery that *causes* a migration is a different job with its own rules, and in a distributed instance it is not done at all - the stack is developed in the origin repo.