sites: raw fetch --scope fetches a page and the subpages its site handler offers as one bundle in incoming/<stem>/, with robots.txt, max_pages and pause_seconds; 10.0.0-beta.6

Files changed:
- CHANGES.md
- VERSION
- docs/how-to/ingest-a-source.md
- docs/how-to/write-a-site-handler.md
- docs/reference/commands/raw-material-and-uploads.md
- docs/reference/configuration.md
- raw/CONTRACT.md
- sites/CONTRACT.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/commands/raw/fetch.py
- tools/chemenu/config_shapes.py
- tools/chemenu/sites/__init__.py
- tools/chemenu/sites/data.py
- tools/chemenu/sites/handlers/learn-microsoft-powershell-module.json
- tools/chemenu/sites/site.schema.json
- tools/chemenu/tests/test_raw_fetch.py
- tools/chemenu/tests/test_raw_fetch_scope.py
- tools/chemenu/tests/test_site_files.py
- tools/chemenu/tests/test_sites.py
- tools/chemenu/web_capture.py

Co-Authored-By: Claude Code <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2

docs/reference/configuration.md

@@ -413,8 +413,8 @@ Size and quota are your deployment's decision, not defaults worth copying blindl
 *site handler* fetches the page in the website's standard form, leaves the page furniture out of the
 derived text and prints hints about the page as `CHECK` lines; today there are two, both for
 `learn.microsoft.com`: `learn-microsoft` for every page of the host and
-`learn-microsoft-powershell-module` for a PowerShell module's overview page, which adds a hint
-naming the cmdlet pages it lists. Handlers are JSON files: the stack's, and the instance's own under `sites/`
+`learn-microsoft-powershell-module` for a PowerShell module's overview page, which offers the cmdlet
+pages it lists as subpages for `raw fetch --scope`. Handlers are JSON files: the stack's, and the instance's own under `sites/`
 (see [How to write a site handler](../how-to/write-a-site-handler.md)).
 The `sites` section is what the instance decides about the handlers, and it is optional: without it every
 handler applies, in the locale `en-us`.
@@ -432,6 +432,9 @@ handler applies, in the locale `en-us`.
   once; an entry whose `path` is left out applies to the whole host. A line is plain text and is
   output as it is, so it is for a reminder to the reader of the capture, never for an instruction
   the tool should carry out.
+- `max_pages` and `pause_seconds` bound `raw fetch --scope`: the most subpages one page may offer
+  before the fetch is refused, and the pause before each subpage request, which keeps the load on
+  the website low.
 
 <!-- wikitool:reference-config-sites -->
 | Field | Tier | Type | Required | Default | Description |
@@ -439,6 +442,8 @@ handler applies, in the locale `en-us`.
 | `disabled` | instance | array of strings | no | `[]` | The names of the site handlers that do not apply. `raw fetch` then treats a page of that website as it does any other. A name no handler has is an error when `raw fetch` reads the section, so a typo cannot leave a handler on. |
 | `locale` | instance | string | no | `"en-us"` | The locale a handler for a multi-language website fetches, in the website's own spelling and lower case. An empty value means the default. |
 | `notes` | instance | array of objects | no | `[]` | Hint lines of the instance's own, printed as `CHECK` lines after a fetch or an `--html` derivation of a page of that host. Each entry has `host` (exact, lower case), an optional `path` (a regular expression matched against the start of the URL path) and `lines` (one to 3 single-line strings of at most 300 characters). A host and path pattern appears once. |
+| `max_pages` | instance | positive integer | no | `200` | The most subpages `raw fetch --scope` fetches for one page. A page that offers more is refused before the first subpage is requested. A number written as a string is accepted. |
+| `pause_seconds` | instance | number | no | `1` | The pause, in seconds, before each subpage request of `raw fetch --scope`. `0` means none. A number written as a string is accepted. |
 
 - **Absent:** Every handler applies, with the locale `en-us` and no further hint lines.
 - **Malformed:** An error when `raw fetch` reads it: it stops with exit 1 before fetching anything. A broken section is never read as "no handlers".
@@ -452,6 +457,8 @@ Example:
       "learn-microsoft-powershell-module"
     ],
     "locale": "en-us",
+    "max_pages": 200,
+    "pause_seconds": 1,
     "notes": [
       {
         "host": "learn.microsoft.com",