raw derive/fetch/pending: Länge als dritte Größenachse beim Ingest - eine CHECK-Zeile nennt jeden geschriebenen Text über LONG_TEXT_CHARS (Zeichen der .md wie geschrieben), raw pending markiert eine lange .md/.txt mit long:; ingest-large-tree liest ihn in Teilen nach der Gliederung, je Teil nur den Zeilenbereich, Extrakt als Faktenliste mit Fundstelle und Modalität (#226); 10.1.0-beta.10
CI / verify (push) Successful in 7m55s
CI / pwsh (push) Successful in 2m41s
Converters / converters (push) Successful in 9m32s
Release / converters (push) Successful in 27s
Release / release (push) Successful in 37s
Release / wiki-mirror (push) Skipped
CI / verify (push) Successful in 7m55s
CI / pwsh (push) Successful in 2m41s
Converters / converters (push) Successful in 9m32s
Release / converters (push) Successful in 27s
Release / release (push) Successful in 37s
Release / wiki-mirror (push) Skipped
Files changed: - CHANGES.md - VERSION - docs/how-to/ingest-a-source.md - docs/reference/commands/raw-material-and-uploads.md - instructions/ingest-large-tree.md - instructions/wiki-ingest/SKILL.md - raw/CONTRACT.md - tools/CONTRACT.md - tools/chemenu/commands/raw/_common.py - tools/chemenu/commands/raw/derive.py - tools/chemenu/commands/raw/fetch.py - tools/chemenu/commands/raw/pending.py - tools/chemenu/tests/test_raw_derive.py - tools/chemenu/tests/test_raw_fetch.py - types/source.guidance.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SnAJ7Z3CpVD3PRbN73QtU2
This commit is contained in:
1 parent
9cb981a4cd
commit
eae4a427d2
15 files changed
+341
-37
No files matched your search
@@ -417,6 +417,15 @@ them is [read-page-images.md](../instructions/read-page-images.md). A PDF withou
|
||||
markings, images and unmatched pages is derived as before. Reading the layout needs `pdfminer.six` beside
|
||||
`pypdf`.
|
||||
|
||||
**A text too long for one pass is named.** After a success `raw derive` prints one more `CHECK`
|
||||
line naming every `.md` it wrote that is longer than one ingest reads in one pass. Length is
|
||||
counted in characters of the file as written, header included, not in tokens: characters are the
|
||||
same for every model, tokens depend on its tokenizer. `raw fetch` prints the same line for the
|
||||
text it writes, and `raw pending` marks a lone `.md` or `.txt` in `incoming/` the same way, since
|
||||
no derivation stands in front of it to print one. The value is the tool's, and `raw derive -h`
|
||||
names it; what an ingest does with such a text is
|
||||
[ingest-large-tree.md](../instructions/ingest-large-tree.md).
|
||||
|
||||
**The original stays.** The derivation is the tool's reading of it, never the source: with the
|
||||
original kept, a claim in `kb/` can still be checked against what the document said when a
|
||||
derivation dropped something, and a later version of the tool can derive it better. Both go into
|
||||
|
||||
Reference in new issue
Block a user