feat(raw): raw derive unpacks ZIP archives and mails with attachments into a folder
A ZIP becomes incoming/<stem>/ with its files and a derived text beside each document, the ZIP is removed and its sha256 named in the output. A mail with attachments becomes a folder with original.eml, derived.md and the attachments; a ZIP attached to a mail is unpacked once, an attached mail is quoted. Unsafe entries, nested ZIPs and size limits are refused before anything is written. Files changed: - CHANGES.md - VERSION - docs/how-to/ingest-a-source.md - docs/reference/commands/README.md - docs/reference/commands/raw-material-and-uploads.md - raw/CONTRACT.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/commands/raw_cmd.py - tools/chemenu/filters/__init__.py - tools/chemenu/filters/archive.py - tools/chemenu/filters/mail.py - tools/chemenu/tests/test_portability.py - tools/chemenu/tests/test_raw_derive.py Co-Authored-By: Claude Code <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
1 parent
9c0bb6bfbf
commit
6659ca5409
14 files changed
+828
-94
No files matched your search
+1
-1
@@ -112,7 +112,7 @@ tools/
|
||||
kb_state.py the KB version (.wikitool-kb.json) and the migration chain
|
||||
corpus_diff.py invariant comparison of kb/ between two revisions
|
||||
web_capture.py `raw fetch`'s core: fetch a page, decide its charset, derive Markdown-like text from the HTML - standard library only, deterministic
|
||||
filters/ the import filters behind `raw derive`: a registry (`__init__.py`) and one module per format family (HTML, mail, Office), each registering the extensions it reads - standard library, or a package imported lazily so a missing one fails only its own format
|
||||
filters/ the import filters behind `raw derive`: a registry (`__init__.py`) and one module per format family (HTML, mail, Office, ZIP), each registering the extensions it reads - standard library, or a package imported lazily so a missing one fails only its own format
|
||||
repo_capture.py `raw capture`/`raw status`'s core: resolve a ref rule, fetch it into a bare cache, select files by glob as blobs, the `_capture.json` manifest - git with the host's credentials, never a prompt; plus the one write back, a single-file commit pushed without force for `export guidelines`
|
||||
guideline_export.py `export guidelines`' core: select pages by `search` predicates, render them into one deterministic GUIDELINES.md, and recognise an export again (`EXPORT_MARKER`) - with no CLI attached
|
||||
search/ pluggable search backends, plus service.py - the search core
|
||||
|
||||
Reference in new issue
Block a user