raw derive PDF: Auszeichnungen auf ein Minimum - Struktur zuerst (Fließtext, Code, Tabellengitter, Vorlagen), ~~ ** * nur wo die Form eindeutig ist, Bildseiten-CHECK nur für Inhaltsbilder (#231); 10.1.0-beta.5

Files changed:
- CHANGES.md
- VERSION
- docs/how-to/ingest-a-source.md
- docs/reference/commands/raw-material-and-uploads.md
- raw/CONTRACT.md
- tools/CONTRACT.md
- tools/chemenu/commands/raw/derive.py
- tools/chemenu/filters/pdf.py
- tools/chemenu/filters/pdf_layout.py
- tools/chemenu/tests/binary_formats.py
- tools/chemenu/tests/test_raw_derive_binary.py

Co-Authored-By: Claude Code <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2

docs/how-to/ingest-a-source.md

@@ -46,11 +46,12 @@ pick a directory:
   sheets land beside the text as tab-separated files, with their formulas listed apart. A ZIP is
   unpacked into a folder instead, and a mail's attachments land beside it. A scanned PDF has no
   text to derive - the tool warns, and there is no OCR. A PDF with an outline gets it in the
-  header of the text, each entry with its page and the line where that page starts; text over a
-  coloured background, in another colour than the running text, or struck through is marked in
-  the text (`<mark data-farbe="gelb">`, `*…*`, `~~…~~`) with a legend in the header, and what a
-  colour means is for the reader to take from the document itself. Pages with pictures are
-  named in a `CHECK` line, because the pictures are not in the text.
+  header of the text, each entry with its page and the line where that page starts; struck-through
+  text, highlighted, bold or clearly differently coloured text, and italic text are marked in the
+  text (`~~…~~`, `**…**`, `*…*`) with a legend in the header, and what a colour means is for the
+  reader to take from the document itself. The tool marks sparingly, in running text and
+  not in code or tables. Pages with pictures or drawings that carry content are named in a `CHECK`
+  line, because the pictures are not in the text.
 - **A page behind a paywall or a login** is saved from your browser into `incoming/`, HTML only. The
   agent derives the same text from that file with `raw fetch --html`. For a website you read often,
   sign in once instead and let the wiki fetch it with your session:

docs/reference/commands/raw-material-and-uploads.md

@@ -193,7 +193,7 @@ tools/wikitool raw derive incoming/scans.zip
 - A folder is derived in place: each file with a filter gets `<stem>.md` beside it, nothing is renamed, and the files without a filter are named in the output.
 - `derived.md` starts with a fixed header (`derived_by`, `filter`, `engine`, `derived`, `derived_from`, `original_name` for a renamed file, `title`, then the fields of the format) followed by the derived text. The same bytes always give the same text; only `derived` differs between runs. `engine` names the tool version and, where a package reads the format, its version.
 - Formats: web pages (`.html`), Word (`.docx`), PowerPoint (`.pptx`), Excel (`.xlsx`, `.xlsm`), e-mail (`.eml`) and ZIP archives (`.zip`), read with the standard library only; PDF (`.pdf`, `pypdf` and `pdfminer.six`) and Outlook messages (`.msg`, `olefile`), each read by packages in `tools/requirements.txt` that are imported only when such a file is derived. A missing package refuses that format alone, with the preflight hint; `wikitool doctor` shows one `import-filter-<name>` line per package-backed format. A PDF gets `<!-- page N -->` markers and a WARN when it yields almost no text (a scan; there is no OCR); an encrypted PDF is refused. A `.msg` is a container like an `.eml`; an embedded message inside it is named, not read. The text of a mail (`.eml` and `.msg` alike) is its plain text, unless that is under 10% of the length of the HTML text - a "plain text version not available" placeholder - in which case it is the HTML text and a WARN names both lengths. The other version of the text in a `multipart/alternative` is not an attachment.
-- A PDF also gets, where the document has them: an `outline` header field (one entry per outline item: title, level, page and the line of `derived.md` where that page's marker is), and markings in the text that follow the visible form only - `<mark data-farbe="<name>">…</mark>` for text over a coloured background, `*…*` for text in a colour other than the running text that is not under a link, `~~…~~` for strike-through. A `markings` header field is the legend (form, colour, places, pages); removing the markings gives the text the PDF has without them. A CHECK names the pages with images, which the text does not contain. A PDF without outline, markings and images is derived as before.
+- A PDF also gets, where the document has them: an `outline` header field (one entry per outline item: title, level, page and the line of `derived.md` where that page's marker is), and markings in the text that follow the visible form only, in plain Markdown: `~~…~~` for strike-through, `**…**` for a highlighter stroke, bold type or a colour clearly unlike the running text (not under a link), `*…*` for italic. Bold, italic and colour count in running text only, never in program text, a table's grid or a shape the document repeats on its pages. A `markings` header field is the legend (at most three lines: form, places, pages; the `**` line names the highlighter colours); removing the markings gives the text the PDF has without them, and a tag never starts or ends on whitespace. A CHECK names the pages with a picture or drawing the text does not contain - not a background, a filler or a picture repeated across the document. A PDF without outline, markings and images is derived as before.
 - A workbook (`.xlsx`, `.xlsm`) gets its cells as files: beside its `.md` a folder `<md stem>.sheets/` (`derived.sheets/` for a single file) holds one `NN-<sheet>.tsv` per sheet - tab-separated per RFC 4180, UTF-8, row 1 of the file is row 1 of the sheet - and, for a sheet with formulas, `NN-<sheet>.formulas.tsv` listing cell, formula and last stored value. The `.md` is the overview: one section per sheet with its range, the file names and a preview. Formulas are not recalculated; a shared formula is written out per cell, and one that cannot be is named `shared:<master cell>` with a WARN. The folder counts against the path budget and is accepted with the rest; an existing one is never overwritten.
 - A ZIP and a mail are containers. A ZIP at the top of `incoming/` becomes the folder `incoming/<stem>/` holding its files under their own names, each with a `<stem>.md` where a filter reads it; the ZIP is removed and the output names it with its sha256. A mail becomes a folder with `original.eml`, `derived.md` and its attachments under their own names; a ZIP attached to it is unpacked once into a folder of its own, and an attached mail is quoted in `derived.md`, not unpacked. A ZIP inside a ZIP, an absolute or `..` path, a symbolic link, an encrypted entry, more than 1000 entries, a file over 25 MiB or more than 250 MiB in total refuse the whole call. Hidden entries (`__MACOSX/`, `.DS_Store`, anything starting with a dot) are named, not written. A name taken twice, or reserved (`original.*` and `derived.md` at the top of a mail's folder, `_capture.json` anywhere), gets `-2`, `-3` before its extension, and the output lists each rename.
 - A name already taken under `raw/` is a WARN, not a refusal: whether the folder is a later edition of that source or a separate one is the human's call. A later edition of a derived source is replaced file by file with `raw accept --replaces`; there is no `--replaces-bundle` for it.