Files
chemenu/instructions/bug-report.md
T
torben 76d67e45ba
CI / verify (push) Successful in 2m5s
Release / release (push) Successful in 39s
feat: bug-report collector pseudonymises identities in two stages, opt-in via --pseudonymise (#158)
Files changed:
- .gitea/workflows/ci.yml
- CHANGES.md
- INSTALL.md
- VERSION
- instructions/bug-report.md
- reports/CONTRACT.md
- tools/README.md
- tools/bugreport.py
- tools/chemenu/tests/test_bugreport.py
2026-09-30 19:03:59 +02:00

186 lines
11 KiB
Markdown

---
type: types/instruction.md
name: bug-report
description: How to collect a bug-report bundle when the stack misbehaves on this machine - run tools/bugreport.py, write a fact-only chronology, tell the user what the bundle contains, and stop short of sending it anywhere.
manual: true
---
# Collect a bug report
When setup, an upgrade or a command fails on one machine and works on another, the person who has to
fix it sees nothing of what happened here. This procedure produces one bundle that answers the first
round of their questions - which machine, which Python, which shell, which harness, what the stack
looked like, what `wikitool` printed - so that the report is not a guessing game.
**Run this only when asked, by name, or when the user agrees to it after a failure.** It is
`manual: true` on purpose: the bundle contains private data, and whether to produce one is the
user's decision, not the agent's. Nothing links to this file from `AGENTS.md` or a skill, apart from
the pointers at the failure decision points of [setup-instance.md](setup-instance.md) and
[upgrade-instance.md](upgrade-instance.md), which only offer it.
<!-- wikitool:toc -->
## Contents
- [What the bundle holds](#what-the-bundle-holds)
- [When to run](#when-to-run)
- [Steps](#steps)
- [Chronology template](#chronology-template)
- [Decision points](#decision-points)
- [Scope](#scope)
<!-- /wikitool:toc -->
## What the bundle holds
`tools/bugreport.py` is a standalone script. It uses the standard library only and imports nothing
from the stack, so it still runs when `wikitool` does not start - no venv, a broken package, a Python
that is too old for the stack. It writes `reports/bugreport-<UTC stamp>/` and a zip beside it, in
four layers:
| Layer | Files | Holds |
|-------|-------|-------|
| 1 Environment | `environment.json` | OS, every Python and shell found, harness, `PATH`, environment variable names (values only for a fixed list), git configuration, venv, line endings, on Windows also long paths, execution policy and mark-of-the-web |
| 2 Stack | `stack.json`, `tree-structure.json` | `VERSION`, the `.wikitool-*.json` files with secrets removed, git status and the last commits, and the shape of `kb/` and `raw/` (counts, depths, path lengths, names that break on Windows) |
| 3 wikitool | `wikitool/*.txt`, `trace.jsonl` | Verbatim output of `version show`, `doctor`, `budget status`, `instructions verify` and `docs verify`, and the caller's session trace. If `version show` fails, `wikitool` counts as not started and nothing else runs |
| 4 Chronology | `CHRONOLOGY.md`, `transcripts/` | What the agent did and saw, and harness transcripts if the user asked for them |
`MANIFEST.md` lists every file and marks the ones that may contain page content and titles: the trace,
the chronology and the transcripts.
Two rules hold for everything the script generates itself. **Secrets are always removed**: values of
keys that look like a token, password, secret, key or auth entry, credentials in URLs, and every value
of that kind found while collecting is also replaced wherever else it turns up. **Page titles are kept
out** unless `--titles` is given: paths under `kb/` and `raw/` are replaced by their shape
(depth, length, whether they hold a space or a non-ASCII character) and `[[wikilinks]]` by the same
flags. A title that stands as bare prose is not found - which is why the trace, the chronology and the
transcripts are marked instead.
**Pseudonymisation is optional** (`--pseudonymise`) and runs in two stages. The bundle then keeps the
*shape* of every name - length per word, spaces, hyphens, character classes, separators, depth of a
path - because that is what an installation failure turns on, and replaces the name itself.
- **Stage 1 is mechanical.** The script reads what the machine knows about its user - user name,
host, home and repository path, git identity, remote URLs - and replaces each identity in every
text file by a placeholder. The same word always gets the same placeholder, in every file and in
JSON-escaped form too. The stack's own public origin and system folder names stay readable.
- **Stage 2 is a model's judgement, applied mechanically.** Names the script cannot know - people,
companies, customers, internal hosts, projects - are named by you as candidates in a file; the
script applies them with the same machinery. You replace nothing in the bundle yourself.
- **Three local files** sit beside the bundle directory, never inside it and never in the zip:
`bugreport-<stamp>.pseudonyms.json` (the mapping), `bugreport-<stamp>.review.txt` (what stage 1
left behind, for you to read) and the candidate file you write. All three contain originals.
- **A residual uncertainty remains and is always named:** stage 2 can miss a name, above all in the
free text of a large trace or transcript that you did not read in full.
The model of the harness reads the bundle for stage 2 - the same place that already sees this
session. The bundle does not leave that place because of it.
## When to run
- Setup or an upgrade failed and the user wants to report it.
- A command fails in a way that looks tied to the machine, and the user asks for a report.
- The user asks for a bug report by name.
## Steps
1. **Tell the user what is about to happen**, in the instance's KB language: a bundle will be written
under `reports/`, it is not pseudonymised, it holds machine, user and path names and the git
remotes, and it is not sent anywhere. Ask whether the session trace, page titles and transcripts
may go in. The defaults are: trace in, titles out, no transcripts.
Ask which channel the bundle will take, and recommend pseudonymisation for every channel except a
direct handover to the maintainer over a secure channel: a tracker issue, an email or a chat is
not one. The default is off; say so, and that it costs one more step.
2. **Write the chronology** to a file under `reports/` (which is gitignored) from the
[template](#chronology-template) below. Facts only - no diagnosis. If the session's own history is
too long to reconstruct, say what is missing instead of filling the gap.
3. **Run the collector**, with whichever Python 3.8 or later the machine has:
```bash
python3 tools/bugreport.py --chronology reports/chronology.md
```
Add `--no-trace` if the user declined the trace, `--titles` if titles may stay,
`--transcript <file>` (repeatable) for a transcript the user pointed at, `--session <id>` to take a
trace other than the caller's, `--pseudonymise` if the user chose it (stage 1). `python3` may be `python` or `py -3` on Windows. If no Python starts
at all, that is the report: give the user the exact error text, verbatim.
4. **Stage 2, only if the bundle was pseudonymised.** Read, in the bundle: `CHRONOLOGY.md` and
`MANIFEST.md` completely; the review list `reports/bugreport-<stamp>.review.txt` completely; the
trace and each transcript completely only if the file is under 100 KB, otherwise only what the
review list points to. Name what is still left of a person, company, customer, internal host or
domain, or project - one per line in `reports/candidates.txt`; `#` starts a comment. Then run:
```bash
python3 tools/bugreport.py --bundle reports/bugreport-<stamp> --candidates reports/candidates.txt
```
The script applies the candidates with the machinery of stage 1, reports which it did not apply
(too short, system vocabulary, not found) and packs the zip again. **Never replace anything in
the bundle yourself.** The run can be repeated with further candidates. It refuses, with exit 1 and
an unchanged bundle, when the mapping beside the bundle is gone.
5. **Do not imitate the collector's session id.** It runs its counting `wikitool` calls under
`WIKITOOL_SESSION_ID=bugreport-<stamp>` itself; do not export that variable, or any `bugreport-*`
one, in the session. The exception it gets in [gates.md](gates.md) § "Taking a new session id"
belongs to the script alone.
6. **Report the result.** Quote the bundle path, the archive path and the privacy notice the script
prints, name the gaps in `MANIFEST.md`, and tell the user to read the bundle before sharing it.
After stage 2, name the residual uncertainty in words of your own: stage 2 is a model's judgement
and can have missed names, above all in the free text of a trace or transcript over 100 KB.
Say that the mapping, the review list and the candidate file hold originals and stay on this
machine, and that the mapping may be deleted after the last stage 2 run. Then stop: the channel - a tracker issue, an email, a chat - is the user's choice, and the agent
never uploads the bundle.
## Chronology template
```markdown
# Chronology
## Goal
One sentence: what the user asked for, e.g. "Set up a new instance from a fresh clone."
## Environment as the agent saw it
Harness, shell, OS, anything the user said about the machine that the collector cannot know.
## Steps
1. `<exact command>` - exit code, and the first line of the error or output that mattered, verbatim.
2. ...
## Expected and observed
- Expected: what the instruction said would happen.
- Observed: what happened instead, verbatim where it is short.
## Changes made by hand
Every file edited or created outside a `wikitool` command, and every setting changed, in order.
## Not known
What the agent could not find out, or did not check.
```
Facts only: no page content, no guessed cause, no advice.
## Decision points
- **The mapping is gone before stage 2?** Stage 2 refuses: it needs the salt stage 1 used, or it would
replace a word differently from the path it already replaced. Collect the report again with
`--pseudonymise`.
- **The user wants a name added after stage 2?** Run stage 2 again with a candidate file holding only
that line. The same run also works when the human spots a name while reading the bundle.
- **The user declines the trace?** Run with `--no-trace`. The manifest records the exclusion.
- **The failure can be reproduced, and there is no trace?** A distributed instance records none by
default. Offer to repeat the failing step in one new shell with `WIKI_TRACE=1` set for that
session only, then collect. Do not change the checkout's telemetry configuration for it.
- **The user wants the report to name pages?** Run with `--titles`; the bundle then also carries
`tree-paths.txt`, which lists every path under `kb/` and `raw/`.
- **The collector exits 1?** It could not write the bundle (a missing input file, a full disk). Read
the message, fix the cause, retry once, then report the exact error.
- **A `wikitool` output in the bundle looks wrong or refuses to run?** Do not re-run it to see more.
The bundle records what happened; that is the report.
## Scope
Covers producing the bundle. It does not cover reading a bundle someone else sent, triaging the
report, or filing it - all of that is the maintainer's side and the user's choice of channel.