torbenandClaude Code
0f39cfdb49
Files changed: - .gitignore - AGENTS.md - CHANGES.md - VERSION - docs/explanation/architecture.md - docs/how-to/README.md - docs/how-to/write-a-site-handler.md - docs/reference/commands/instance-health.md - docs/reference/commands/raw-material-and-uploads.md - docs/reference/configuration.md - instructions/dev/doc-pull-through.md - raw/CONTRACT.md - sites/CONTRACT.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/commands/dist_cmd.py - tools/chemenu/commands/docs_verify.py - tools/chemenu/commands/doctor.py - tools/chemenu/commands/git_publish.py - tools/chemenu/commands/raw/fetch.py - tools/chemenu/ownership.py - tools/chemenu/sites/__init__.py - tools/chemenu/sites/data.py - tools/chemenu/sites/handlers/learn-microsoft-powershell-module.json - tools/chemenu/sites/handlers/learn-microsoft.json - tools/chemenu/sites/learn_microsoft.py - tools/chemenu/sites/site.schema.json - tools/chemenu/tests/test_dist_cmd.py - tools/chemenu/tests/test_doctor.py - tools/chemenu/tests/test_site_files.py - tools/chemenu/toc.py Co-Authored-By: Claude Code <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
2.9 KiB
Write a site handler
Teach tools/wikitool raw fetch how one website serves its pages: the standard form of its URLs,
the page furniture to leave out of the derived text, and a hint or two for whoever reads the
capture. A handler is a JSON file in sites/ at the root of the instance - data, not a program -
so every clone fetches the website the same way.
The stack already ships handlers, for learn.microsoft.com among others. They are the same kind of
file, so the quickest start is to copy one:
cp tools/chemenu/sites/handlers/learn-microsoft.json sites/my-website.json
The file name without .json is the handler's name: lower-case letters, digits and hyphens.
1. Find out what the website does
Fetch a page of the website and read the derived .md:
tools/wikitool raw fetch <url>
Two things are worth a handler:
- The same page under several URLs - a language segment, a tracking parameter. A rewrite turns every spelling into one standard form, so two people ingesting the page get the same source.
- Page furniture in the derived text - a sign-in notice, an action bar, a feedback footer. Open
the saved
.html, find the element that holds it, and note itsid,classor an attribute.
2. Write the file
A small handler that drops a cookie banner and a footer on one host:
{
"host": "docs.example.org",
"summary": "Drops the cookie banner and the feedback footer.",
"drop": [
{"id": "cookie-banner"},
{"class": "feedback", "inside": "footer"}
]
}
host is required and matched exactly, in lower case. path narrows the handler to a part of the
website; among the handlers of a host the longest matching path wins. The other fields - rewrites,
checks, parent - and what each selector may say are in sites/CONTRACT.md.
3. Try it
Fetch a page again, or derive a saved one without a fetch:
tools/wikitool raw fetch <url>
tools/wikitool raw fetch --html <saved.html> --url <url>
The header of the derived text then says site: <name> (instance), and the furniture is gone. A
check shows up as a CHECK line when it applies to the page.
tools/wikitool doctor lists the handlers of the instance and fails on a file that cannot be used,
naming the file and the reason. Such a file also stops a raw fetch of its own host with exit 1,
before anything is fetched or written, so a handler that did not apply never leaves furniture in a
source unnoticed.
Replace or extend a stack handler
A handler of the instance for the same host and path as a stack handler replaces it. To keep
what the stack knows and add to it, name it as parent and give the new file a path of its own.
To switch a handler off without deleting it, list its name under disabled in the sites section
of the settings - see Configuration.