|
|
|
@@ -0,0 +1,94 @@
|
|
|
|
|
---
|
|
|
|
|
type: types/instruction.md
|
|
|
|
|
name: corpus-policy
|
|
|
|
|
description: What "curated enough" means for kb/ when it is demo and testbed at once, the measurable floors that define it, and what a reactive fix to the corpus may and may not do.
|
|
|
|
|
---
|
|
|
|
|
# Keep kb/ curated enough to develop against, without a second corpus
|
|
|
|
|
|
|
|
|
|
This instance runs one `kb/` for two purposes at once: a public demo and the testbed this stack
|
|
|
|
|
is developed against. There is deliberately no fixture corpus, no `--with-demo` export, and no
|
|
|
|
|
second repository - see Gitea #28. The corpus's size and shape are set by what targeted
|
|
|
|
|
development needs, not by a synthetic fixture size or a demo aesthetic.
|
|
|
|
|
|
|
|
|
|
## When to run
|
|
|
|
|
|
|
|
|
|
- Before judging whether the corpus can exercise a change under development - ranking, index
|
|
|
|
|
scaling, orphan detection, a new label, a new type-spec.
|
|
|
|
|
- Before a reactive fix touches `kb/` content rather than the failing code - the floors below
|
|
|
|
|
are what decides whether the fix may proceed as-is.
|
|
|
|
|
- Picking up Gitea #28 or #30, or any issue that references this file.
|
|
|
|
|
|
|
|
|
|
## The floors
|
|
|
|
|
|
|
|
|
|
Each is mechanically checkable with an existing `wikitool` command; none needs new tool code.
|
|
|
|
|
A floor exists to keep some class of bug observable, not to describe an aesthetic target - so
|
|
|
|
|
when a session is about to make one of these numbers *worse*, that is the signal to stop and
|
|
|
|
|
think, not a number to defend for its own sake.
|
|
|
|
|
|
|
|
|
|
| Floor | Check | Why this number |
|
|
|
|
|
|---|---|---|
|
|
|
|
|
| Every page type has ≥1 page | `wikitool search --field type=types/<t>.md` | A type with zero pages means its schema, its collection contract and its lint rules are unexercised |
|
|
|
|
|
| Every declared subtype has ≥1 page | `wikitool search --field <x>_type=<v>` | Same reasoning, one level down - `entity_type`, `concept_type`, `source_type` |
|
|
|
|
|
| ≥5 pages corpus-wide with ≥3 `sources:` entries | one-off script, see below | Provenance fan-in - multiple sources backing one claim - is a real case only a handful of pages exercise; fewer than 5 and a provenance-index bug can hide |
|
|
|
|
|
| Orphan pages (no inbound link) between 1 and 10 | `wikitool lint` | Zero orphans makes orphan detection itself unobservable; more than 10 means the corpus stopped being curated |
|
|
|
|
|
| Average outbound wikilinks per page ≥4 | one-off script, see below | Below this, ranking and graph-traversal work has too little structure to exercise |
|
|
|
|
|
|
|
|
|
|
A floor is a lower bound only. There is no upper bound on page count or on any of these numbers
|
|
|
|
|
except the orphan ceiling above - a corpus that outgrows these floors through real ingests is
|
|
|
|
|
not a problem this file cares about.
|
|
|
|
|
|
|
|
|
|
**Measured 2026-09-03** (see Gitea #28): 181 pages, 14/14 types and subtypes covered, 12 pages
|
|
|
|
|
with ≥3 sources, 3 orphans, 6.2 average outbound links. All floors held without any manufactured
|
|
|
|
|
content - the corpus was already big enough when the question was asked.
|
|
|
|
|
|
|
|
|
|
A type or subtype sitting at exactly the floor - one page - shows no set-level bugs, only that
|
|
|
|
|
the type is *reachable*. That is a soft target for the next `wiki-ingest` that happens to
|
|
|
|
|
produce a matching page, never a reason to write one: filing an unsourced page to clear a floor
|
|
|
|
|
is exactly what AGENTS.md invariant 3 forbids, floor or no floor. The same holds for an
|
|
|
|
|
authorised link label with zero live uses (`wikitool xref` reports these) - fill it when a real
|
|
|
|
|
edge calls for it, never manufacture one to exercise the label.
|
|
|
|
|
|
|
|
|
|
To check the two floors without a dedicated command, walk `kb/**/*.md` (excluding
|
|
|
|
|
`INDEX.md`/`COLLECTION.md`/`CONTRACT.md`/`CONVENTIONS.md`), parse frontmatter, and: count pages
|
|
|
|
|
whose `related:` array (resolved against page titles) has ≥3 entries for outbound density; count
|
|
|
|
|
`sources:` array length ≥3 for the provenance floor. `wikitool search` and `wikitool lint`
|
|
|
|
|
cover everything else in the table.
|
|
|
|
|
|
|
|
|
|
## What a reactive fix may do to kb/ content
|
|
|
|
|
|
|
|
|
|
Three tiers, by how much of the corpus a change touches:
|
|
|
|
|
|
|
|
|
|
1. **Pointwise - always allowed.** Creating, updating, renaming or deleting a single page
|
|
|
|
|
through the normal tools (`new`, `touch`, the page-lifecycle procedure), below the
|
|
|
|
|
Mass-Update Gate's threshold. This is ordinary work and needs no special permission.
|
|
|
|
|
2. **Corpus-wide - planned only, never reactive.** A migration, a vocabulary sweep, a bulk
|
|
|
|
|
`touch` across many pages. This needs its own issue and, per `work/CONTRACT.md`, a `work/`
|
|
|
|
|
run - never a same-session reaction to whatever the session was originally doing. If a
|
|
|
|
|
session hits the Mass-Update Gate (exit 42, see `instructions/gates.md`) while working on
|
|
|
|
|
something else, it does not fetch the `--confirm` token to push through: it stops, opens an
|
|
|
|
|
issue for the corpus-wide change, and finishes the original task without it.
|
|
|
|
|
3. **Reactive - never allowed.** Deleting or reshaping a page to make a failing test pass;
|
|
|
|
|
restructuring corpus content to route around a tool bug (AGENTS.md invariant 7); using
|
|
|
|
|
`kb/` as a scratch surface for a tool experiment. If a stack change under development needs a
|
|
|
|
|
corpus shape that does not exist, build it as a pytest fixture (see the next section) -
|
|
|
|
|
never manufacture it in `kb/`.
|
|
|
|
|
|
|
|
|
|
## Relationship to the test fixtures
|
|
|
|
|
|
|
|
|
|
`tools/chemenu/tests/conftest.py`'s `kb_dir`/`raw_dir` fixtures and `test_pipeline_l0.py` cover
|
|
|
|
|
the **small, isolated** case: a handful of pages, built fresh per test, hermetic. `kb/` covers
|
|
|
|
|
the **large, connected** case: 181+ pages, grown link density, real provenance history that no
|
|
|
|
|
per-test fixture reconstructs economically. The cut: if a `tmp_path` tree can reproduce what the
|
|
|
|
|
test needs, it belongs in a fixture; if the test needs density or scale that only a grown corpus
|
|
|
|
|
has, it belongs against `kb/`. Neither absorbs the other's job - see
|
|
|
|
|
[testing-conventions.md](testing-conventions.md).
|
|
|
|
|
|
|
|
|
|
## Decision points
|
|
|
|
|
|
|
|
|
|
- **A floor would be violated by an in-progress change - is that a blocker?** Only for the
|
|
|
|
|
orphan ceiling and the type/subtype floors, since those two can go to zero. The density and
|
|
|
|
|
provenance floors move gradually with ordinary ingests and are not gating on any single
|
|
|
|
|
session.
|
|
|
|
|
- **Corpus is "too small" for a feature under development?** That is not this file's problem to
|
|
|
|
|
solve by adding pages - see tier 3 above. Either the feature waits for a real ingest to supply
|
|
|
|
|
the shape, or it gets a pytest fixture.
|
torben