sites: raw fetch --scope fetches a page and the subpages its site handler offers as one bundle in incoming/<stem>/, with robots.txt, max_pages and pause_seconds; 10.0.0-beta.6

Files changed:
- CHANGES.md
- VERSION
- docs/how-to/ingest-a-source.md
- docs/how-to/write-a-site-handler.md
- docs/reference/commands/raw-material-and-uploads.md
- docs/reference/configuration.md
- raw/CONTRACT.md
- sites/CONTRACT.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/commands/raw/fetch.py
- tools/chemenu/config_shapes.py
- tools/chemenu/sites/__init__.py
- tools/chemenu/sites/data.py
- tools/chemenu/sites/handlers/learn-microsoft-powershell-module.json
- tools/chemenu/sites/site.schema.json
- tools/chemenu/tests/test_raw_fetch.py
- tools/chemenu/tests/test_raw_fetch_scope.py
- tools/chemenu/tests/test_site_files.py
- tools/chemenu/tests/test_sites.py
- tools/chemenu/web_capture.py

Co-Authored-By: Claude Code <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2

sites/CONTRACT.md

@@ -23,6 +23,7 @@ as `site: <name> (instance)`.
 | `rewrites` | Optional, at most 5. Each `{"pattern", "replacement"}` is applied to the URL once, in order, as `re.sub(pattern, replacement, url, count=1)`; `{locale}` in a replacement is the `locale` of the `sites` section. The result is the standard form that is fetched; the URL as typed goes into the header as `requested_url` |
 | `drop` | Optional, at most 40. Selectors for the page furniture to leave out of the derived text; the `.html` stays as received |
 | `checks` | Optional, at most 3 - counting the parent's. Hint lines printed as `CHECK` when they apply to the page |
+| `subpages` | Optional. Which links of a page are its subpages, so `raw fetch --scope` can fetch the page and them as one bundle. A child inherits the parent's block unless it has its own |
 
 A selector in `drop`, and the `count` of a check, is an object with any of `tag`, `id`, `class`
 (one token of the element's classes), `attr` (an attribute that must be present, equal to `value`
@@ -35,6 +36,14 @@ it applies only when at least one element matches the selector, and `{count}` is
 neither, it applies to every page of the handler. A text that uses `{meta}` or `{count}` must have
 the field that fills it.
 
+`subpages` is `{"links", "prefix"?}`. `links` is a selector, as above, for the `<a>` elements whose
+`href` names a subpage; `prefix` limits the path a subpage may have: `{page}` (the default) is the
+path of the page itself, with a trailing slash, and a value starting with `/` is a literal path.
+A link is resolved against the page's URL, loses its fragment and counts only on the page's own
+host, under the prefix, and once; the page itself is never its own subpage. Nothing is fetched
+because of it unless the user runs `raw fetch <url> --scope`; without it the page prints one
+`CHECK` line with the number of subpages. At most `max_pages` of the `sites` section are fetched.
+
 `tools/chemenu/sites/site.schema.json` is the exact shape; every field outside the table above is
 refused.