chemenu/instructions/migrate-corpus.md

---
type: types/instruction.md
name: migrate-corpus
description: Change the shape of every kb/ page at once - a schema field, a vocabulary, a language - in planned units, with a mechanical check per unit and a recorded KB version at the end.
---
# Migrate the corpus

A change that touches the *shape* of pages rather than their content: a new required
frontmatter field, a renamed enum value, a section heading vocabulary, a language. It is not an
ingest and not a lint fix - nothing new is learned, the same knowledge is restated in a new
form. The failure mode is therefore specific and quiet: **something present before is missing
afterwards**, and the corpus is still internally consistent, so `lint` reports nothing.

Every rule below was paid for once already. The German translation of 248 pages found four
defects this way - a dropped citation that silently unsourced a claim, a dropped wikilink, an
invented one, and a translated H1 - and three of the four had unchanged link *sets* and only
changed counts.

<!-- wikitool:toc -->
## Contents

- [When to run](#when-to-run)
- [Steps](#steps)
- [Decision points](#decision-points)
- [Writing the migration document](#writing-the-migration-document)
- [Scope](#scope)
<!-- /wikitool:toc -->

## When to run

A change that would otherwise be applied to more than a handful of pages by hand, or any change
declared by a migration document under `instructions/migrations/`. A single page is
`wiki-manage`; a raw tree is [ingest-large-tree.md](ingest-large-tree.md).

## Steps

1. **Open a workshop.** `tools/wikitool work new --key <slug>` - not `ingest-`, which is
   reserved for keys derived from a `raw/` path. `plan.md` cuts the corpus into units and says
   why each cut falls where it does; `README.md` carries the closing condition and the
   decisions made so far. See [work/CONTRACT.md](../work/CONTRACT.md).

2. **Size the units by the iteration budget, not by feel.** One unit costs roughly
   `N × touch` + `index rebuild` + `log append` + `publish` (twice - the Mass-Update Gate
   refuses once and publishes on the confirm), plus `sources rebuild-index` if it contains
   source pages. Against the 60-call ceiling that puts the ceiling near 55 pages; aim for 48 or
   fewer.

   **Units and publishes are not the same boundary.** The budget is per session id; the gate is
   per publish. Several units may run back to back, each with its own
   `WIKITOOL_SESSION_ID="<slug>/u<N>"`, and publish once together - which is what the written
   unit boundaries in `plan.md` make legitimate rather than a way around a gate refusal (see
   [gates.md](gates.md)).

3. **Rewrite the unit's pages.** Bodies only. Frontmatter is written with `touch`, never by
   hand, and never by a subagent.

4. **Check mechanically, before anything else:**

   ```bash
   tools/wikitool migrate verify --from HEAD --path kb/<area> --fail-on-error
   ```

   This is the step the whole procedure exists for. It compares wikilink and citation
   **counts**, footnote definitions, H1 and structural frontmatter against the last commit.
   Run it before the summaries, before `lint`, before anything - it is the cheapest place to
   catch a subagent that helpfully translated a link target.

5. **Write the summaries yourself** with `touch --summary`, from the original. Never paste a
   subagent's proposal unread: they embellish, and a summary is a claim about the page.

6. **`index rebuild`, then `lint` - and read the whole report**, not only the sections this
   unit could plausibly have touched. The translation's first unit had a frontmatter
   round-trip bug that surfaced as a schema error on a field nobody had edited.

7. **`log append`, then publish** through [publish-cycle.md](publish-cycle.md). Expect exit 42
   on a corpus-sized change; reproduce the breakdown for the user and wait.

8. **Carry the vocabulary between units.** Terminology settled in unit 5 and re-decided in unit
   9 is the failure a glossary file in the workshop exists to prevent. Add to it *before*
   dispatching the next unit.

9. **Record the migration** once the last unit is published:

   ```bash
   tools/wikitool migrate done <version> --pages <N>
   ```

   This advances `kb_version` in `.wikitool-kb.json`. It refuses any version that is not the
   next link in the chain, so a multi-step upgrade cannot silently skip one.

10. **Close the workshop** per [work/CONTRACT.md](../work/CONTRACT.md), after promoting whatever
    outlives the run. The translation's glossary became
    [german-terminology.md](german-terminology.md); its checklist and unit plan died with the
    directory, correctly.

## Decision points

- **Can the change be made backwards-compatible instead?** Prefer it. A vocabulary migration
  does not need a flag day: `tools/chemenu/sections.py` gives each heading one canonical name
  and any number of aliases, so a page is found under the old name and takes the new one only
  when it is rewritten. Removing an alias afterwards is a second breaking change, not a cleanup.
- **Mechanical or assisted?** A rename with a fixed rule is `mechanical` and wants a script; a
  change needing a judgment call per page is `assisted` and wants this procedure. There is no
  `migrate run` today - `migration_kind` describes the work, it does not perform it.
- **The check finds something mid-unit.** Fix it in that unit and re-run `verify`. Never carry
  a finding into the next unit "to fix later": the next unit's diff baseline is this unit's
  commit, so an uncorrected drop becomes invisible.
- **Contradiction with an existing page.** Never overwrite. Record both, ask the user, and pull
  the confidence down with `touch --confidence-base` if it stays unresolved.

## Writing the migration document

A migration that a distributed instance must also run is a `manual: true` instruction under
`instructions/migrations/<version>-<slug>.md`, carrying `migrates_to:` and `migration_kind:`.
`tools/wikitool migrate status` builds the outstanding chain from those files, and `version
bump` refuses a compatibility-breaking release that has none.

Write it for a reader who has the new machinery and the old content, and who is not you: what
changed, which pages are affected, how to tell a migrated page from an unmigrated one, and what
`migrate verify` should report when it is done.

**Baseline: 1.0.0.** Migrations that predate it - the type-system move, the `confidence_base`
backfill, the German section headings, the translation itself - have no documents and will not
get any. An instance older than that is re-exported, not migrated.

## Scope

For `kb/` content. A single page is `wiki-manage`; a `raw/` tree is
[ingest-large-tree.md](ingest-large-tree.md). Changing the machinery that *causes* a migration
is a different job with its own rules, and in a distributed instance it is not done at all -
the stack is developed in the origin repo.