feat(raw): raw derive unpacks ZIP archives and mails with attachments into a folder
A ZIP becomes incoming/<stem>/ with its files and a derived text beside each document, the ZIP is removed and its sha256 named in the output. A mail with attachments becomes a folder with original.eml, derived.md and the attachments; a ZIP attached to a mail is unpacked once, an attached mail is quoted. Unsafe entries, nested ZIPs and size limits are refused before anything is written. Files changed: - CHANGES.md - VERSION - docs/how-to/ingest-a-source.md - docs/reference/commands/README.md - docs/reference/commands/raw-material-and-uploads.md - raw/CONTRACT.md - tools/CONTRACT.md - tools/README.md - tools/chemenu/commands/raw_cmd.py - tools/chemenu/filters/__init__.py - tools/chemenu/filters/archive.py - tools/chemenu/filters/mail.py - tools/chemenu/tests/test_portability.py - tools/chemenu/tests/test_raw_derive.py Co-Authored-By: Claude Code <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
1 parent
9c0bb6bfbf
commit
6659ca5409
14 files changed
+828
-94
No files matched your search
@@ -276,6 +276,19 @@ a stem of well over a hundred characters, where keeping the file's own name twic
|
||||
as `original_name`. A folder already in `incoming/` is derived in place instead: each file with a
|
||||
filter gets a `<stem>.md` beside it, and nothing is renamed.
|
||||
|
||||
**A container is unpacked into its folder.** A ZIP becomes the folder `incoming/<stem>/` holding
|
||||
its files under their own names, each with a filter next to a `<stem>.md`, and the ZIP itself is
|
||||
removed - the tool's output names it and its sha256, and naming it on the source page is a human's
|
||||
step. A mail with attachments becomes a folder holding `original.eml`, `derived.md` and the
|
||||
attachments under their own names; a ZIP attached to a mail is unpacked once into a folder of its
|
||||
stem, and a mail attached to a mail is quoted in `derived.md`, not unpacked. A ZIP inside a ZIP,
|
||||
and any entry that is an absolute or `..` path, a symbolic link or encrypted, is refused, as is
|
||||
an archive past the entry, file or total size limit that `raw derive -h` names. Hidden entries
|
||||
(`__MACOSX/`, `.DS_Store`, anything starting with a dot) are not written but are named in the
|
||||
output. A name that is taken or reserved (`original.*` and `derived.md` at the top of a mail
|
||||
folder, `_capture.json` anywhere) gets `-2`, `-3` before its extension, and the output lists
|
||||
every rename.
|
||||
|
||||
**The original stays.** The derivation is the tool's reading of it, never the source: with the
|
||||
original kept, a claim in `kb/` can still be checked against what the document said when a
|
||||
derivation dropped something, and a later version of the tool can derive it better. Both go into
|
||||
|
||||
Reference in new issue
Block a user