torben
76d67e45ba
Files changed: - .gitea/workflows/ci.yml - CHANGES.md - INSTALL.md - VERSION - instructions/bug-report.md - reports/CONTRACT.md - tools/README.md - tools/bugreport.py - tools/chemenu/tests/test_bugreport.py
83 lines
4.8 KiB
Markdown
83 lines
4.8 KiB
Markdown
# reports/ - Generated Output
|
|
|
|
Derived output that must stay out of git. Three kinds live here:
|
|
|
|
- **Lint reports**, written by `tools/wikitool lint --markdown "reports/Lint Report
|
|
<YYYY-MM-DD>.md"`.
|
|
- **Traces**, under `reports/telemetry/<session>/trace.jsonl` - the append-only record of
|
|
what a session did, written by `wikitool` itself and by the harness hooks. See
|
|
[../EVALS.md](../EVALS.md) for the event contract and what is redacted.
|
|
- **Bug-report bundles**, under `reports/bugreport-<UTC stamp>/` and a zip beside each, written by
|
|
`tools/bugreport.py` on request - see [../instructions/bug-report.md](../instructions/bug-report.md).
|
|
|
|
**Everything in this directory except this file is gitignored.** A lint report is a derived
|
|
copy of recomputable truth: its structural sections can be regenerated from the tree at any
|
|
commit, so committing them would create a second, drifting copy of something the tool already
|
|
answers on demand.
|
|
|
|
## The durable half
|
|
|
|
A report also carries a `## Semantic Review` section, which is the LLM's judgment and *cannot*
|
|
be regenerated. Because the file itself is never committed, that judgment must be carried out
|
|
of here before the pass ends:
|
|
|
|
- Findings that change a page belong in the page, via `wikitool touch` / `xref` / `new`.
|
|
- The summary of the pass belongs in `kb/log.md`, via
|
|
`tools/wikitool log append --op lint`.
|
|
|
|
A lint pass that leaves its conclusions only in `reports/` has lost them.
|
|
|
|
## Traces
|
|
|
|
A trace is neither recomputable nor durable: rerunning a session produces a different one, and
|
|
nothing else in the repo can reconstruct it. It is still gitignored, because it records
|
|
prompts and assistant replies in cleartext and belongs to the machine it ran on, not to the
|
|
repository. What a trace concludes - a scored eval, a failure taxonomy - is carried out the
|
|
same way a lint report's judgment is: into `kb/`, `work/`, or `kb/log.md`.
|
|
|
|
`WIKI_TRACE_DIR` redirects the tree; `WIKI_TRACE` turns recording on or off, overriding both the
|
|
installation-form default (on for a dev checkout, off for a distributed instance -
|
|
`chemenu.telemetry.policy`) and a per-checkout `.wikitool-telemetry.json`; `WIKI_TRACE_CONTENT=0`
|
|
keeps lengths and digests instead of text. See [../EVALS.md](../EVALS.md) § "Whether it runs at
|
|
all" for the full precedence and both quantity caps below.
|
|
|
|
## Bug-report bundles
|
|
|
|
A bundle is a snapshot of one machine at one moment, made so that someone else can read what
|
|
happened here. It is not recomputable and not durable, and it **contains private data**: machine,
|
|
user and path names, `PATH` entries, git remotes and commit subjects, and - when included - the
|
|
session trace, the chronology and transcripts, which may hold page content and titles. Secrets are
|
|
removed by the collector; the rest is the reader's to check before a bundle leaves the machine.
|
|
Nothing uploads it: the channel is the user's choice.
|
|
|
|
With `--pseudonymise` the collector replaces known identities by consistent, shape-preserving
|
|
placeholders, and `--bundle`/`--candidates` applies further names a model found. Three files then
|
|
sit **beside** the bundle directory, never inside it and never in its zip, and all three contain
|
|
originals: `bugreport-<stamp>.pseudonyms.json` (the mapping and its salt),
|
|
`bugreport-<stamp>.review.txt` (what stage 1 left behind) and the candidate file the agent writes.
|
|
`MANIFEST.md` lists the placeholders, never an original. What stage 2 finds is a model's judgement,
|
|
so the manifest and the collector's closing output name a residual uncertainty.
|
|
|
|
The collector's two counting `wikitool` calls (`instructions verify`, `docs verify`) run under the
|
|
session id `bugreport-<stamp>`; its budget-exempt ones inherit the caller's. A
|
|
`reports/telemetry/bugreport-*` directory is therefore that run's trace and belongs to no session
|
|
of yours.
|
|
|
|
## Retention
|
|
|
|
**Lint reports: none.** Old ones are local scratch; delete them freely. There is nothing to
|
|
retire with `wikitool rm`, because no report is ever a wiki page - `lint-report` is a
|
|
contract-only type-spec with no `base_dir:` and cannot be instantiated under `kb/`.
|
|
|
|
**Bug-report bundles: none.** Delete them freely once they have been read or sent; nothing refers
|
|
to one afterwards. The mapping beside a pseudonymised bundle is needed until the last stage 2 run,
|
|
because stage 2 refuses without it; after that it may be deleted, and it should be, since it holds
|
|
the originals.
|
|
|
|
**Traces: two enforced caps, applied by the writer itself, never by a separate cleanup pass.**
|
|
A byte cap per session trace (default 5 MiB, `WIKI_TRACE_MAX_SESSION_BYTES`) and a retention
|
|
limit on the number of session directories under `reports/telemetry/` (default 250,
|
|
`WIKI_TRACE_KEEP_SESSIONS`), both fail-silent like the writer itself. Neither ever removes a
|
|
session directory in bulk - only the `trace.jsonl`/`.limit` files a cap's own rule names, and
|
|
only once the directory is otherwise empty.
|