sites: the instance writes its own site handlers as JSON files under sites/, the stack's learn.microsoft.com handlers become data files in the same format (#210); 10.0.0-beta.4
Files changed: - .gitignore - AGENTS.md - CHANGES.md - VERSION - docs/explanation/architecture.md - docs/how-to/README.md - docs/how-to/write-a-site-handler.md - docs/reference/commands/instance-health.md - docs/reference/commands/raw-material-and-uploads.md - docs/reference/configuration.md - instructions/dev/doc-pull-through.md - raw/CONTRACT.md - sites/CONTRACT.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/commands/dist_cmd.py - tools/chemenu/commands/docs_verify.py - tools/chemenu/commands/doctor.py - tools/chemenu/commands/git_publish.py - tools/chemenu/commands/raw/fetch.py - tools/chemenu/ownership.py - tools/chemenu/sites/__init__.py - tools/chemenu/sites/data.py - tools/chemenu/sites/handlers/learn-microsoft-powershell-module.json - tools/chemenu/sites/handlers/learn-microsoft.json - tools/chemenu/sites/learn_microsoft.py - tools/chemenu/sites/site.schema.json - tools/chemenu/tests/test_dist_cmd.py - tools/chemenu/tests/test_doctor.py - tools/chemenu/tests/test_site_files.py - tools/chemenu/toc.py Co-Authored-By: Claude Code <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
1 parent
2213d68900
commit
0f39cfdb49
31 files changed
+916
-124
No files matched your search
@@ -10,6 +10,7 @@ Working with the wiki:
|
||||
|
||||
- [Ingest a source](ingest-a-source.md) - a file, a folder, a web page, or a document submitted
|
||||
through the MCP server
|
||||
- [Write a site handler](write-a-site-handler.md) - teach `raw fetch` how a website serves its pages
|
||||
- [Capture documentation from a git repository](capture-repository-docs.md) - and keep it current
|
||||
- [Export guidelines to captured repositories](export-guidelines.md)
|
||||
- [Query the wiki](query-the-wiki.md) - asking, and searching yourself
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
# Write a site handler
|
||||
|
||||
Teach `tools/wikitool raw fetch` how one website serves its pages: the standard form of its URLs,
|
||||
the page furniture to leave out of the derived text, and a hint or two for whoever reads the
|
||||
capture. A handler is a JSON file in `sites/` at the root of the instance - data, not a program -
|
||||
so every clone fetches the website the same way.
|
||||
|
||||
The stack already ships handlers, for `learn.microsoft.com` among others. They are the same kind of
|
||||
file, so the quickest start is to copy one:
|
||||
|
||||
```bash
|
||||
cp tools/chemenu/sites/handlers/learn-microsoft.json sites/my-website.json
|
||||
```
|
||||
|
||||
The file name without `.json` is the handler's name: lower-case letters, digits and hyphens.
|
||||
|
||||
## 1. Find out what the website does
|
||||
|
||||
Fetch a page of the website and read the derived `.md`:
|
||||
|
||||
```bash
|
||||
tools/wikitool raw fetch <url>
|
||||
```
|
||||
|
||||
Two things are worth a handler:
|
||||
|
||||
- **The same page under several URLs** - a language segment, a tracking parameter. A rewrite
|
||||
turns every spelling into one standard form, so two people ingesting the page get the same source.
|
||||
- **Page furniture in the derived text** - a sign-in notice, an action bar, a feedback footer. Open
|
||||
the saved `.html`, find the element that holds it, and note its `id`, `class` or an attribute.
|
||||
|
||||
## 2. Write the file
|
||||
|
||||
A small handler that drops a cookie banner and a footer on one host:
|
||||
|
||||
```json
|
||||
{
|
||||
"host": "docs.example.org",
|
||||
"summary": "Drops the cookie banner and the feedback footer.",
|
||||
"drop": [
|
||||
{"id": "cookie-banner"},
|
||||
{"class": "feedback", "inside": "footer"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`host` is required and matched exactly, in lower case. `path` narrows the handler to a part of the
|
||||
website; among the handlers of a host the longest matching path wins. The other fields - `rewrites`,
|
||||
`checks`, `parent` - and what each selector may say are in [sites/CONTRACT.md](../../sites/CONTRACT.md).
|
||||
|
||||
## 3. Try it
|
||||
|
||||
Fetch a page again, or derive a saved one without a fetch:
|
||||
|
||||
```bash
|
||||
tools/wikitool raw fetch <url>
|
||||
tools/wikitool raw fetch --html <saved.html> --url <url>
|
||||
```
|
||||
|
||||
The header of the derived text then says `site: <name> (instance)`, and the furniture is gone. A
|
||||
check shows up as a `CHECK` line when it applies to the page.
|
||||
|
||||
`tools/wikitool doctor` lists the handlers of the instance and fails on a file that cannot be used,
|
||||
naming the file and the reason. Such a file also stops a `raw fetch` of its own host with exit 1,
|
||||
before anything is fetched or written, so a handler that did not apply never leaves furniture in a
|
||||
source unnoticed.
|
||||
|
||||
## Replace or extend a stack handler
|
||||
|
||||
A handler of the instance for the same host and path as a stack handler **replaces** it. To keep
|
||||
what the stack knows and add to it, name it as `parent` and give the new file a `path` of its own.
|
||||
|
||||
To switch a handler off without deleting it, list its name under `disabled` in the `sites` section
|
||||
of the settings - see [Configuration](../reference/configuration.md#site-handlers).
|
||||
Reference in new issue
Block a user