raw derive/fetch/pending: Länge als dritte Größenachse beim Ingest - eine CHECK-Zeile nennt jeden geschriebenen Text über LONG_TEXT_CHARS (Zeichen der .md wie geschrieben), raw pending markiert eine lange .md/.txt mit long:; ingest-large-tree liest ihn in Teilen nach der Gliederung, je Teil nur den Zeilenbereich, Extrakt als Faktenliste mit Fundstelle und Modalität (#226); 10.1.0-beta.10

Files changed:
- CHANGES.md
- VERSION
- docs/how-to/ingest-a-source.md
- docs/reference/commands/raw-material-and-uploads.md
- instructions/ingest-large-tree.md
- instructions/wiki-ingest/SKILL.md
- raw/CONTRACT.md
- tools/CONTRACT.md
- tools/chemenu/commands/raw/_common.py
- tools/chemenu/commands/raw/derive.py
- tools/chemenu/commands/raw/fetch.py
- tools/chemenu/commands/raw/pending.py
- tools/chemenu/tests/test_raw_derive.py
- tools/chemenu/tests/test_raw_fetch.py
- types/source.guidance.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2

instructions/ingest-large-tree.md

@@ -1,7 +1,7 @@
 ---
 type: types/instruction.md
 name: ingest-large-tree
-description: Ingest a raw tree too large, or a single source too broad, for one pass - in planned units through a work/ workshop, instead of one oversized source page or a cohort of stub pages.
+description: Ingest a raw tree too large, or a single source too broad or too long, for one pass - in planned units through a work/ workshop, instead of one oversized source page, a cohort of stub pages or a text read in one go.
 ---
 # Ingest a large raw tree
 
@@ -10,7 +10,9 @@ read, promoted and published on its own. The plan and the intermediate extracts
 `work/` workshop, so the run survives across sessions and days instead of having to fit in one.
 
 A single source carrying too many subjects lands here too, and takes the workshop but not the
-cut - see [A broad source is not cut](#a-broad-source-is-not-cut).
+cut - see [A broad source is not cut](#a-broad-source-is-not-cut). So does a single source too
+long to read in one pass, which keeps its one raw file and is read in parts - see
+[A long source is read in parts](#a-long-source-is-read-in-parts).
 
 <!-- wikitool:toc -->
 ## Contents
@@ -18,6 +20,7 @@ cut - see [A broad source is not cut](#a-broad-source-is-not-cut).
 - [When to run](#when-to-run)
 - [Tiers](#tiers)
 - [A broad source is not cut](#a-broad-source-is-not-cut)
+- [A long source is read in parts](#a-long-source-is-read-in-parts)
 - [Steps](#steps)
 - [Decision points](#decision-points)
 - [Scope](#scope)
@@ -25,7 +28,7 @@ cut - see [A broad source is not cut](#a-broad-source-is-not-cut).
 
 ## When to run
 
-Any one of these is enough, on either axis.
+Any one of these is enough, on any of the three axes.
 
 **Volume** - more material than one pass can read:
 
@@ -40,9 +43,26 @@ Any one of these is enough, on either axis.
   concepts together**. Estimate it from the reading, before writing anything; on a finished page
   the same number is the length of `entities:` plus `concepts:`.
 
-The two axes take different routes through this procedure. Volume is cut: several files become
-several units, each its own source page. Breadth is not cut at all - it takes the workshop for
-the extract pass and nothing else (§ [A broad source is not cut](#a-broad-source-is-not-cut)).
+**Length** - one text longer than one pass reads well:
+
+- `raw derive` or `raw fetch` printed the `CHECK` line naming the text as longer than one ingest
+  reads in one pass, or `raw pending` listed the entry with `long:`. For a `.md` or `.txt` that
+  is not in `incoming/`, compare `wc -m <file>` with the value `tools/wikitool raw derive -h`
+  names.
+
+Length is an axis of its own because neither of the others fires for one long document on a
+few subjects, and reading it whole costs more than it seems. A session has already loaded a
+large fixed context before it opens the source - the harness's own, `AGENTS.md`, the skill, the
+contracts and conventions an ingest reads. Summaries of long input lose far more by leaving facts
+out than by getting them wrong, and small models lose recall well before their context window is
+full. Summarising the summaries of the parts does not recover it either. So the text is read in
+parts, and what each part yields is a list of facts, not a summary.
+
+The three axes take different routes through this procedure. Volume is cut: several files become
+several units, each its own source page. Breadth and length are not cut - one raw file keeps one
+source page. Breadth takes the workshop for the extract pass
+(§ [A broad source is not cut](#a-broad-source-is-not-cut)); length takes it for reading in parts
+and an extract per part (§ [A long source is read in parts](#a-long-source-is-read-in-parts)).
 
 Otherwise use `wiki-ingest` unchanged. This procedure costs a workshop and a planning round;
 a single, narrow document does not earn it.
@@ -54,6 +74,7 @@ a single, narrow document does not earn it.
 | Standard | One file, or a small folder | `wiki-ingest`, unchanged |
 | Tree | A volume trigger above | This instruction |
 | Broad | The breadth trigger above | This instruction, § A broad source is not cut |
+| Long | The length trigger above | This instruction, § A long source is read in parts |
 | Audited | A unit covering secrets, RBAC, ingress, disaster recovery, or an audit trail | This instruction plus step 5c |
 
 ## A broad source is not cut
@@ -91,6 +112,25 @@ produces when every name in the source is turned into a page: pages that restate
 pass `lint` (which measures structure, never substance), and read as covered ground to the next
 session.
 
+## A long source is read in parts
+
+A long source is one raw file, so the two rules of
+[A broad source is not cut](#a-broad-source-is-not-cut) hold for it unchanged: `raw/` keeps it
+whole, and it gets one source page. The cut is in the reading, never in the file. What the
+workshop buys:
+
+1. `tools/wikitool work new --input <the file, or the folder raw derive made>`, then one unit per
+   **part of the text** in `plan.md` - step 3 says where a long source is cut.
+2. Per unit, steps 5a and 5b: read the unit's line range and nothing else, and extract a fact
+   list.
+3. One source page, one publish: steps 5d-5e run once, after the last unit, over all the
+   extracts. The page is written from the fact lists of every unit - never from summaries of
+   the units, which is the summary of summaries this route exists to avoid.
+
+**Long and broad at once?** The same workshop, units by the text's parts. Which subjects earn a
+page - point 3 of the broad route - is decided over the collected extracts, after the last unit,
+not unit by unit.
+
 ## Steps
 
 1. **Survey the tree, do not read it yet.**
@@ -129,6 +169,17 @@ session.
    planned page title, and the reason for the cut. Record what is excluded from the run
    entirely, and why.
 
+   **A long source is cut by its outline**, not by subject: a unit is a page range, cut at the
+   highest outline level whose sections each stay under the length the `CHECK` line names. A
+   section longer than that - a long annex, a commentary part - is cut at the next level down,
+   or at page boundaries where it has none. A derived PDF's `outline` header field gives each
+   entry's page and the line of that page's marker, so the ranges come from the header, not from
+   reading the text. Without an outline the units are page ranges between the
+   `<!-- page N -->` markers (`grep -n '<!-- page ' <file>`); a text without those either - a
+   fetched page, a plain text - is cut at its own headings (`grep -n '^#' <file>`), else at line
+   numbers. Measure a range with `sed -n '<from>,<to>p' <file> | wc -m`. For each unit record its
+   title, its pages and its **line range** in `plan.md`: that range is all step 5a reads.
+
    Then fill the `README.md` checklist - one line per unit.
 
 4. **Agree the plan with the user.** This is the one decision checkpoint for the whole run:
@@ -144,11 +195,21 @@ session.
       (AGENTS.md invariant 4). A derived PDF whose header has `images:` or `unmatched:` is
       followed by `read-page-images.md`, whose threshold of 20 pages holds per unit.
 
+      **A long source:** read the unit's line range from `plan.md` and nothing else - the header
+      once, in the first unit. Reading the whole file again in every unit is the one pass this
+      route replaces, repeated. The page images are those of the unit's own pages.
+
    b. **Extract** into `work/<runkey>/extract-u<N>.md`: the hard facts (IPs, ports, versions,
       paths, commands, config values), each with the raw file it came from, plus what is
       *new* relative to what step 1's searches found. Write down what you are dropping and
       why - that becomes the page's `## Not Extracted` section.
 
+      **A long source:** the extract is a **fact list**, one entry per fact, each with where it
+      stands - the page, and the rule or section number where the text has one - and, where the
+      text states them, its condition, its exception, its deadline or number, and its modality
+      (*must*, *should*, *may*, *must not*). A normative text - rules, a regulation, a contract -
+      binds through exactly those parts, and they are what a summary drops first.
+
    c. **Audited tier only:** before touching any existing page, check the extract back against
       the raw files and record findings in `work/<runkey>/audit.md`, each as
       `Status: open` / `Status: resolved` with what changed. Do not promote while a finding is
@@ -157,8 +218,8 @@ session.
 
    d. **Promote** with `wiki-ingest` steps 4-10, using the extract as the input rather than the
       raw files - step 5 (promotion itself) is a no-op here, since the unit's raw file is
-      already under `raw/` (§ [When to run](#when-to-run) named the volume/breadth trigger that
-      put it there). Fill `## Not Extracted` from b.
+      already under `raw/` (§ [When to run](#when-to-run) named the trigger that put it
+      there). Fill `## Not Extracted` from b.
 
       This includes step 4's commitment question, asked once **per unit** rather than once for
       the whole tree: a unit is a subject the same way a single-file `wiki-ingest` source is one,
@@ -190,6 +251,10 @@ session.
 - **The breadth trigger fired, but the source reads narrower than it looked?** Close the
   workshop and run an ordinary `wiki-ingest`. The count is estimated before the reading, so
   being wrong about it is expected; carrying a workshop nobody needs is the avoidable half.
+- **The length trigger fired for a text that is mostly not prose** - a long register, tables of
+  figures, an annex of forms? The units still cut the reading. A unit with nothing to extract is
+  recorded as excluded in `plan.md`, with the reason, and that reason goes into
+  `## Not Extracted`.
 - **A unit turns out to be a duplicate of an existing page?** Update that page instead of
   creating a second one, and say so in `plan.md`. That is a result, not a failure.
 - **The plan changes mid-run?** Edit `plan.md` and the checklist, and say why in `README.md`.
@@ -199,7 +264,7 @@ session.
 
 ## Scope
 
-This is about *size*, on the two axes § When to run names: volume and breadth. It is not about
+This is about *size*, on the three axes § When to run names: volume, breadth and length. It is not about
 difficulty - a short but hard source, a specification that needs careful reading, is still an
 ordinary `wiki-ingest`. And nothing here changes what a page must contain:
 [kb/CONTRACT.md](../kb/CONTRACT.md) and the collection contracts still decide that.

instructions/wiki-ingest/SKILL.md

@@ -42,7 +42,8 @@ validator complains - and the ticked list is the only record that they happened.
 
 ## Steps
 
-1. **Read the source.** Read the file completely, wherever it currently sits - `incoming/` for
+1. **Read the source.** Read the file completely - unless the length check below fires, which
+   it does before anything is read - wherever it currently sits: `incoming/` for
    the normal path, or already under `raw/` when the run started there (a file `capture-session`
    just wrote, for instance, which skips step 5 entirely). A PDF, mail, Office file or ZIP: run
    `tools/wikitool raw derive <path>` first and read the `.md` - its output names what to check.
@@ -165,14 +166,19 @@ validator complains - and the ticked list is the only record that they happened.
    reads often, mention the lasting way as well: fetching with their own session
    (`docs/how-to/fetch-pages-behind-a-login.md`) - set up by the user, not by you.
 
-   **Check the size first, on both axes.** *Volume* - how many raw files this ingest covers -
-   and *breadth* - how many entities and concepts this one source would produce or update.
-   Either one past the thresholds in `instructions/ingest-large-tree.md` § When to
+   **Check the size first, on all three axes.** *Volume* - how many raw files this ingest
+   covers -, *breadth* - how many entities and concepts this one source would produce or update -
+   and *length* - whether one text is longer than one pass reads. Length is known before the
+   reading: `raw derive` and `raw fetch` print a `CHECK` line naming such a text, and
+   `raw pending` lists a lone `.md` or `.txt` with `long:`; a text file named outside
+   `incoming/` is measured with `wc -m <file>` against the value `tools/wikitool raw derive -h`
+   names. Any one past the thresholds in `instructions/ingest-large-tree.md` § When to
    run is that procedure, not this one: stop and follow it. There, volume is cut into units;
-   breadth cannot be cut at all (`raw/` keeps a file whole, and one raw file has one owning
-   source page) and buys an extract pass instead, before any page is written. Skipping either
-   fails silently: an oversized source page drops most of what it read, and an over-broad one
-   leaves a cohort of stub pages behind.
+   breadth and length cannot be cut (`raw/` keeps a file whole, and one raw file has one owning
+   source page) - breadth buys an extract pass before any page is written, length a reading in
+   parts with a fact list per part. Skipping any of them fails silently: an oversized source
+   page drops most of what it read, an over-broad one leaves a cohort of stub pages behind, and
+   a long text read in one go loses its facts by leaving them out.
 
    **A trigger firing here promotes now, ahead of step 4's commitment discussion below - the one
    deliberate exception to this skill's ordering.** `ingest-large-tree.md`'s own step 2
@@ -482,6 +488,11 @@ validator complains - and the ticked list is the only record that they happened.
 - **One source names far more subjects than usual?** That is breadth, not volume. It is not
   split into several sources - it cannot be - and it does not get a page per name either:
   `instructions/ingest-large-tree.md` § A broad source is not cut.
+- **One source is longer than one pass reads?** That is length - the `CHECK` line of
+  `raw derive` or `raw fetch` says so, or `raw pending`'s `long:`. It is not split into several
+  sources and not summarised whole: it is read in parts along its outline, each part yielding a
+  fact list, and one source page is written from all of them -
+  `instructions/ingest-large-tree.md` § A long source is read in parts.
 - **The entry is a new edition of a captured bundle?** Step 1's branch for updating the captured
   repositories is how a run gets here: `raw status` reported it, `raw capture --update` wrote it,
   and `raw accept incoming/<bundle> --replaces-bundle <raw-bundle>` takes it in - the name refusal