raw: fetch behind a login - raw login stores the user's session as a secret outside the tree, fetch.routes sends a host through it or to an external fetcher, a handler's render goes through the optional browser component (preflight --with browser) (#169); 10.0.0-beta.8

Files changed:
- CHANGES.md
- INSTALL.md
- VERSION
- docs/how-to/README.md
- docs/how-to/fetch-pages-behind-a-login.md
- docs/how-to/ingest-a-source.md
- docs/reference/commands/README.md
- docs/reference/commands/instance-health.md
- docs/reference/commands/raw-material-and-uploads.md
- docs/reference/configuration.md
- docs/reference/exit-codes.md
- instructions/preflight.md
- instructions/wiki-ingest/SKILL.md
- raw/CONTRACT.md
- sites/CONTRACT.md
- tools/CONTRACT.md
- tools/README.md
- tools/chemenu/browser.py
- tools/chemenu/cli_contract.py
- tools/chemenu/commands/doctor.py
- tools/chemenu/commands/raw/__init__.py
- tools/chemenu/commands/raw/fetch.py
- tools/chemenu/commands/raw/login.py
- tools/chemenu/config_shapes.py
- tools/chemenu/fetch_routes.py
- tools/chemenu/reference_doc.py
- tools/chemenu/settings.py
- tools/chemenu/sites/__init__.py
- tools/chemenu/sites/data.py
- tools/chemenu/sites/site.schema.json
- tools/chemenu/tests/test_cli.py
- tools/chemenu/tests/test_config_shapes.py
- tools/chemenu/tests/test_doctor.py
- tools/chemenu/tests/test_preflight.py
- tools/chemenu/tests/test_preflight_pwsh.py
- tools/chemenu/tests/test_raw_fetch_routes.py
- tools/chemenu/tests/test_raw_login.py
- tools/chemenu/tests/test_site_files.py
- tools/chemenu/web_capture.py
- tools/chemenu/web_session.py
- tools/preflight.ps1
- tools/preflight.sh
- tools/requirements-browser.txt

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2

sites/CONTRACT.md

@@ -24,6 +24,8 @@ as `site: <name> (instance)`.
 | `drop` | Optional, at most 40. Selectors for the page furniture to leave out of the derived text; the `.html` stays as received |
 | `checks` | Optional, at most 3 - counting the parent's. Hint lines printed as `CHECK` when they apply to the page |
 | `subpages` | Optional. Which links of a page are its subpages, so `raw fetch --scope` can fetch the page and them as one bundle. A child inherits the parent's block unless it has its own |
+| `session` | Optional. For a website behind a login: the page `raw login` opens for the user to sign in on, and how a fetched page shows that it arrived signed in. A child inherits the parent's block unless it has its own |
+| `render` | Optional, `true` or `false` (the default). `true` fetches the handler's pages through a headless browser, because they draw their content with JavaScript; the header then says `rendered: true`. A child inherits `true` from its parent |
 
 A selector in `drop`, and the `count` of a check, is an object with any of `tag`, `id`, `class`
 (one token of the element's classes), `attr` (an attribute that must be present, equal to `value`
@@ -44,6 +46,20 @@ host, under the prefix, and once; the page itself is never its own subpage. Noth
 because of it unless the user runs `raw fetch <url> --scope`; without it the page prints one
 `CHECK` line with the number of subpages. At most `max_pages` of the `sites` section are fetched.
 
+`session` is `{"login", "signed_in"?, "signed_out"?}`. `login` is the `http(s)` URL of the login
+page. `signed_in` is a selector, as above, that matches only on a signed-in page - a logout link,
+an account menu; `signed_out` one that matches only on a page that was not - a paywall teaser, a
+"subscribe to read on" box. At least one of the two is required, and with both, both must hold. A
+page fetched with the user's session that does not show the sign stops `raw fetch` with exit 42
+and writes nothing, so pick a sign that the full article always carries, or its teaser always
+does. The block alone changes nothing: the pages of the host are fetched with the session only
+once the instance routes the host there (`fetch.routes` in the settings). Which website allows
+its pages to be fetched this way is the user's to judge, and a handler records nothing about it.
+
+`render` changes what the `.html` is: the page as the browser drew it, not the bytes the server
+sent. Declare it only for a website whose text is missing from the HTML it sends. `raw login` and
+a rendered fetch need the optional browser component (`tools/preflight.sh --with browser`).
+
 `tools/chemenu/sites/site.schema.json` is the exact shape; every field outside the table above is
 refused.
 
@@ -70,4 +86,6 @@ host does not stop the fetch.
 ## Safety
 
 A pattern runs only against the URL and its path, never against the page. A handler cannot run
-code, read a file or reach another host: it is data, not a program.
+code, read a file or reach another host: it is data, not a program. A `session` holds no
+credential either - only where the user signs in and what a signed-in page looks like; the
+session itself is stored outside the working tree by `raw login`.