sites: raw fetch gets site handlers - standard URL form, per-website cleanup and CHECK hints as a code registry, learn.microsoft.com first, plus a sites settings section (#200); 10.0.0-beta.3
Files changed: - tools/chemenu/sites/ (registry, learn-microsoft handlers), web_capture.py, commands/raw/fetch.py, config_shapes.py - tests, fixture, raw/CONTRACT.md, tools/README.md, docs, CHANGES.md, VERSION Files changed: - CHANGES.md - VERSION - docs/how-to/ingest-a-source.md - docs/reference/commands/README.md - docs/reference/commands/raw-material-and-uploads.md - docs/reference/configuration.md - raw/CONTRACT.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/commands/raw/fetch.py - tools/chemenu/config_shapes.py - tools/chemenu/reference_doc.py - tools/chemenu/sites/__init__.py - tools/chemenu/sites/learn_microsoft.py - tools/chemenu/tests/fixtures/learn_microsoft/oslicense-module.html - tools/chemenu/tests/test_config_shapes.py - tools/chemenu/tests/test_sites.py - tools/chemenu/web_capture.py Co-Authored-By: Claude Code <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
1 parent
298b514119
commit
2213d68900
18 files changed
+1226
-27
No files matched your search
@@ -118,6 +118,7 @@ tools/
|
||||
kb_state.py the KB version (.wikitool-kb.json) and the migration chain
|
||||
corpus_diff.py invariant comparison of kb/ between two revisions
|
||||
web_capture.py `raw fetch`'s core: fetch a page, decide its charset, derive Markdown-like text from the HTML - standard library only, deterministic
|
||||
sites/ the site handlers behind `raw fetch`: a registry (`__init__.py`) and one module per website, each registering the standard form of its URLs, the page furniture to leave out of the derived text, and hint lines - all of it data (rewrites, selectors, text), standard library only
|
||||
filters/ the import filters behind `raw derive`: a registry (`__init__.py`) and one module per format family (HTML, mail, Outlook MSG, Office, PDF, ZIP), each registering the extensions it reads - standard library, or a package imported lazily so a missing one fails only its own format
|
||||
repo_capture.py `raw capture`/`raw status`'s core: resolve a ref rule, fetch it into a bare cache, select files by glob as blobs, the `_capture.json` manifest - git with the host's credentials, never a prompt; plus the one write back, a single-file commit pushed without force for `export guidelines`
|
||||
guideline_export.py `export guidelines`' core: select pages by `search` predicates, render them into one deterministic GUIDELINES.md, and recognise an export again (`EXPORT_MARKER`) - with no CLI attached
|
||||
|
||||
Reference in new issue
Block a user