chemenu/instructions/migrate-corpus.md

type, name, description

type	name	description
types/instruction.md	migrate-corpus	Change the shape of every kb/ page at once - a schema field, a vocabulary, a language - in planned units, with a mechanical check per unit and a recorded KB version at the end.

Migrate the corpus

A change that touches the shape of pages rather than their content: a new required frontmatter field, a renamed enum value, a section heading vocabulary, a language. It is not an ingest and not a lint fix - nothing new is learned, the same knowledge is restated in a new form. The failure mode is therefore specific and quiet: something present before is missing afterwards, and the corpus is still internally consistent, so lint reports nothing.

Every rule below was paid for once already. The German translation of 248 pages found four defects this way - a dropped citation that silently unsourced a claim, a dropped wikilink, an invented one, and a translated H1 - and three of the four had unchanged link sets and only changed counts.

Contents

• When to run
• Steps
• Decision points
• Writing the migration document
• Scope

When to run

A change that would otherwise be applied to more than a handful of pages by hand, or any change declared by a migration document under instructions/migrations/. A single page is wiki-manage; a raw tree is ingest-large-tree.md.

Steps

1. Open a workshop. tools/wikitool work new --key <slug> - not ingest-, which is reserved for keys derived from a raw/ path. plan.md cuts the corpus into units and says why each cut falls where it does; README.md carries the closing condition and the decisions made so far. See work/CONTRACT.md.

2. Size the units by the iteration budget, not by feel. One unit costs roughly N × touch + index rebuild + log append + publish (twice - the Mass-Update Gate refuses once and publishes on the confirm), plus sources rebuild-index if it contains source pages. Against the 60-call ceiling that puts the ceiling near 55 pages; aim for 48 or fewer.

   Units and publishes are not the same boundary. The budget is per session id; the gate is per publish. Several units may run back to back, each with its own WIKITOOL_SESSION_ID="<slug>/u<N>", and publish once together - which is what the written unit boundaries in plan.md make legitimate rather than a way around a gate refusal (see gates.md).

3. Rewrite the unit's pages. Bodies only. Frontmatter is written with touch, never by hand, and never by a subagent.

4. Check mechanically, before anything else:

   tools/wikitool migrate verify --from HEAD --path kb/<area> --fail-on-error
   

   This is the step the whole procedure exists for. It compares wikilink and citation counts, footnote definitions, H1 and structural frontmatter against the last commit. Run it before the summaries, before lint, before anything - it is the cheapest place to catch a subagent that helpfully translated a link target.

5. Write the summaries yourself with touch --summary, from the original. Never paste a subagent's proposal unread: they embellish, and a summary is a claim about the page.

6. index rebuild, then lint - and read the whole report, not only the sections this unit could plausibly have touched. The translation's first unit had a frontmatter round-trip bug that surfaced as a schema error on a field nobody had edited.

7. log append, then publish through publish-cycle.md. Expect exit 42 on a corpus-sized change; reproduce the breakdown for the user and wait.

8. Carry the vocabulary between units. Terminology settled in unit 5 and re-decided in unit 9 is the failure a glossary file in the workshop exists to prevent. Add to it before dispatching the next unit.

9. Record the migration once the last unit is published:

   tools/wikitool migrate done <version> --pages <N>
   

   This advances kb_version in .wikitool-kb.json. It refuses any version that is not the next link in the chain, so a multi-step upgrade cannot silently skip one.

10. Close the workshop per work/CONTRACT.md, after promoting whatever outlives the run. The translation's glossary became german-terminology.md; its checklist and unit plan died with the directory, correctly.

Decision points

• Can the change be made backwards-compatible instead? Prefer it. A vocabulary migration does not need a flag day: tools/chemenu/sections.py gives each heading one canonical name and any number of aliases, so a page is found under the old name and takes the new one only when it is rewritten. Removing an alias afterwards is a second breaking change, not a cleanup.
• Mechanical or assisted? A rename with a fixed rule is mechanical and wants a script; a change needing a judgment call per page is assisted and wants this procedure. There is no migrate run today - migration_kind describes the work, it does not perform it.
• The check finds something mid-unit. Fix it in that unit and re-run verify. Never carry a finding into the next unit "to fix later": the next unit's diff baseline is this unit's commit, so an uncorrected drop becomes invisible.
• Contradiction with an existing page. Never overwrite. Record both, ask the user, and pull the confidence down with touch --confidence-base if it stays unresolved.

Writing the migration document

A migration that a distributed instance must also run is a manual: true instruction under instructions/migrations/<version>-<slug>.md, carrying migrates_to: and migration_kind:. tools/wikitool migrate status builds the outstanding chain from those files, and version bump refuses a compatibility-breaking release that has none.

Write it for a reader who has the new machinery and the old content, and who is not you: what changed, which pages are affected, how to tell a migrated page from an unmigrated one, and what migrate verify should report when it is done.

Baseline: 1.0.0. Migrations that predate it - the type-system move, the confidence_base backfill, the German section headings, the translation itself - have no documents and will not get any. An instance older than that is re-exported, not migrated.

Scope

For kb/ content. A single page is wiki-manage; a raw/ tree is ingest-large-tree.md. Changing the machinery that causes a migration is a different job with its own rules, and in a distributed instance it is not done at all - the stack is developed in the origin repo.