sites: raw fetch --scope fetches a page and the subpages its site handler offers as one bundle in incoming/<stem>/, with robots.txt, max_pages and pause_seconds; 10.0.0-beta.6

Files changed:
- CHANGES.md
- VERSION
- docs/how-to/ingest-a-source.md
- docs/how-to/write-a-site-handler.md
- docs/reference/commands/raw-material-and-uploads.md
- docs/reference/configuration.md
- raw/CONTRACT.md
- sites/CONTRACT.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/commands/raw/fetch.py
- tools/chemenu/config_shapes.py
- tools/chemenu/sites/__init__.py
- tools/chemenu/sites/data.py
- tools/chemenu/sites/handlers/learn-microsoft-powershell-module.json
- tools/chemenu/sites/site.schema.json
- tools/chemenu/tests/test_raw_fetch.py
- tools/chemenu/tests/test_raw_fetch_scope.py
- tools/chemenu/tests/test_site_files.py
- tools/chemenu/tests/test_sites.py
- tools/chemenu/web_capture.py

Co-Authored-By: Claude Code <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2

raw/CONTRACT.md

@@ -270,6 +270,20 @@ resolution and what a broken file does are in [sites/CONTRACT.md](../sites/CONTR
 switches a handler off, sets the locale, and adds hint lines of its own per host and path pattern,
 printed as `CHECK` lines too - at most three per pattern.
 
+**A page that is an index can be fetched together with its subpages.** A handler may name which
+links of its page are subpages (the `subpages` field of [sites/CONTRACT.md](../sites/CONTRACT.md));
+for the module overview of `learn.microsoft.com` those are the cmdlet pages it lists. `raw fetch`
+then prints one `CHECK` line with their number, and nothing more is fetched. `raw fetch <url>
+--scope` fetches the page and its subpages as one bundle - a folder `incoming/<stem>/` holding a
+`.html`/`.md` pair per page, written as for a single page, and `_scope.json`, a manifest of the
+pages with their URL, retrieval time and the SHA-256 of the received HTML. `raw accept
+incoming/<stem>` takes the folder as one source. Only the URLs the handler names are fetched, on
+the host of the page and under the path the handler allows, never a link followed further. The
+`robots.txt` of the host is honoured for the subpages: one it forbids, or one that cannot be read,
+refuses the whole fetch. So do more subpages than `sites.max_pages`, a subpage that fails or is not
+HTML, and a folder of that name in `incoming/` or `raw/`; everything is checked before the first
+write, and a failure leaves `incoming/` as it was.
+
 Whether a capture is a whole article or only its teaser cannot be told mechanically - a teaser
 can be longer than the 200 characters under which `raw fetch` warns. The session that reads the
 `.md` in full is what judges it: text that visibly breaks off ("continue reading with...", a