# Conversation Transcript - Hardening the Test Suite Against Silent Environment Dependencies Session

> Source: Claude Code session (`claude-opus-5`), llm-wiki-test1 workspace
> Collected: 2026-08-31
> Participant: Torben
> Fidelity: **faithful summary transcript, not a verbatim log.** The user's instructions,
> command output, test counts, commit hashes and CI log excerpts are quoted verbatim. The
> assistant's reasoning and the order in which files were read are condensed. All numbers and
> paths below were observed in the session, not reconstructed afterwards.
> No second-hand material: no subagent was spawned, and every claim was verified in-session by
> running the command that shows it.
> No credentials appeared in the session.

Covers the implementation of Gitea issue #8 - an autouse pytest fixture that cuts every test off
from the machine it runs on - shipped as `1.7.1` (`31c9b81`, tag `v1.7.1`), plus the two
follow-up issues the work exposed (#22, #23). The session ran under the `stack-dev` skill
throughout.

---

## Turn 1 - the instruction

> implementiere issue #8

Issue #8, `prio/1` `size/M`, titled "Testsuite gegen stille Umgebungsabhängigkeiten härten". Read
in full along with its one comment, before any code was opened.

### What the issue said

The first CI run that ever reached `pytest` (run 52, 2026-08-30) failed two tests that had been
green on every developer machine for months:

```
FAILED wiki_tools/tests/test_new_page.py::test_new_source_author_falls_back_to_git_config
FAILED wiki_tools/tests/test_provenance.py::test_new_source_with_multiple_raw_files
AssertionError: ERROR No author configured for this instance.
2 failed, 628 passed
```

Cause: `config.default_author()` runs `git config user.name` with `cwd=config.ROOT`. The fixture
root is not a repository, so the answer came from the **global** git configuration of whoever
started pytest. As `root` in the job container there is none.

Both were repaired in `1.0.1`. The issue's own framing of what remained: the suite "hat nicht
gewarnt, sie war einfach grün, weil die Umgebung zufällig passte."

The comment, added 2026-08-31 during the `1.2.0` work on #12, is the load-bearing part:

> Das ist der dritte und vierte Fall derselben Abhängigkeit, geschrieben von jemandem, der das
> Issue vorher gelesen hatte. Die Frage aus dem Issue-Text — „wie viele unbekannte gibt es" —
> ist damit weniger interessant als die andere: **die Suite lädt neue Fälle schneller ein, als
> jemand sie findet.**

Two options were on the table, and the issue had already ranked them: an autouse fixture in
`conftest.py` (preferred), or a second hardened `pytest` step in CI (fallback, "schwächer, weil
es die Abhängigkeit erst nach dem Push meldet").

### The decision, and why the fallback was rejected

The fixture. The comment settles it: a guard that reports after the push loses to a suite that
acquires new cases faster than anyone finds them. A CI-only guard also never protects the
developer's own run, which is where the cases are written.

This was not a close call and was not re-litigated. What the session did add was a reason the
issue could not have known - see Turn 6.

## Turn 2 - measuring before changing

Rather than start from the issue's list, the environment surface was measured directly:

```bash
grep -rn "environ\|getenv" wiki_tools/*.py | grep -v tests/
```

Which yields the tool's own reads: `WIKI_AUTHOR` (`config.py:59`), `WIKITOOL_SESSION_ID`
(`session.py:17`), `WIKITOOL_UPDATE_URL` / `WIKITOOL_UPDATE_TOKEN` (`version.py:50,53`), and the
telemetry set `WIKI_TRACE`, `WIKI_TRACE_DIR`, `WIKI_TRACE_CONTENT`, `WIKI_TRACE_MAX_CONTENT`
(`telemetry/writer.py`, `telemetry/scrub.py`).

Then the baseline, which turned out to matter:

```bash
env -u WIKI_AUTHOR GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_SYSTEM=/dev/null HOME=<empty> \
  .venv/bin/python -m pytest -q
→ 695 passed in 10.30s
```

**The suite was already green under the hardened environment.** The four known cases had all
been patched in `1.0.1` and `1.2.0`, and no fifth existed at that moment. This reframed the
change: it is not a repair, it is a guard - and a guard whose value has to be demonstrated
separately, because "everything still passes" proves nothing about it.

## Turn 3 - what was built

### The fixture

`hermetic_environment`, autouse, in `tools/wiki_tools/tests/conftest.py`, next to the existing
`isolated_trace_dir`. Per test: `HOME` and `XDG_CONFIG_HOME` into that test's own `tmp_path`,
`GIT_CONFIG_GLOBAL` and `GIT_CONFIG_SYSTEM` to `/dev/null`, and two lists cleared.

**Beyond the issue's list**, git's own identity and location variables were added: `GIT_DIR`,
`GIT_WORK_TREE`, `GIT_AUTHOR_NAME`, `GIT_AUTHOR_EMAIL`, `GIT_COMMITTER_NAME`,
`GIT_COMMITTER_EMAIL`, `EMAIL`. The reasoning, recorded in the code comment: `GIT_AUTHOR_NAME`
outranks `git config user.name`, so it is literally the same failure the issue describes through
a different door; and a stray `GIT_DIR` would point every fixture repo at the developer's own
checkout.

### The ordering problem between two autouse fixtures

`hermetic_environment` clears `WIKI_TRACE`. `isolated_trace_dir` sets `WIKI_TRACE_DIR`. If the
clearing ran second it would not touch `WIKI_TRACE_DIR`, so the suite would still work - but the
correctness would rest on pytest's declaration order rather than on anything stated.

`isolated_trace_dir` now takes `hermetic_environment` as a parameter. Not for its value, for the
ordering. Both docstrings say so.

`WIKI_TRACE_DIR` is deliberately the one variable left **set**. Acceptance criterion 4 of the
issue: tracing must never be disabled suite-wide, because two telemetry tests assert that a
trace gets written.

### What was deliberately not done

**No `WIKI_AUTHOR` in the fixture base.** This was the cheaper fix and the wrong one, and the
issue had pre-empted it in acceptance criterion 2: a shared default would make
`default_author()`'s `None` branch untestable, because that branch only exists on a machine that
knows nobody. The suite would look greener and prove less.

**`test_new_source_fails_hard_without_any_author` keeps patching `default_author` directly.**
Under the fixture the environment would now resolve to `None` on its own, so the patch could
have been deleted as redundant. It was kept: the patch pins the value regardless of what the
environment does, which keeps the test about the CLI's error path rather than about the
environment. Criterion 3 asked exactly this and the reasoning is recorded in
`testing-conventions.md`'s decision points.

## Turn 4 - proving the guard actually guards

A fixture nothing asserts against can be weakened or lose a variable, and everything stays green
until the next run on a foreign machine - the same failure one level up. So
`tools/wiki_tools/tests/test_hermetic_env.py` was written: the cleared variables, the empty
`HOME` inside `tmp_path`, that `git config user.name` answers nothing outside a repo with a local
identity, that tracing stays on and redirected.

Plus all three branches of `config.default_author()`, including the `None` branch **that the
hardening makes writable for the first time**.

The counter-proof that this means something, run in-session:

```
default_author() with the ambient environment: 'Torben Nehmer'
```

Without the fixture, `default_author()` in a non-repository returns the developer's global git
identity. With it, `None`. The new test would have failed before the change and passes after -
which is the demonstration the Turn 2 baseline could not provide.

## Turn 5 - verification across four environments

Three locally, all producing the identical count:

| Environment | Before | After |
|---|---|---|
| Developer shell | green | `702 passed` |
| Deliberately poisoned: `WIKI_AUTHOR`, `WIKI_TRACE=0`, `WIKITOOL_*`, `GIT_*` set to junk | untested | `702 passed` |
| `env -i`, empty `HOME`, no git configuration | **red** (historically) | `702 passed` |

Plus `docs verify`, `instructions verify`, `lint --fail-on-error`, `instructions sync` - all OK.

The poisoned run is the one the issue did not ask for. It tests the opposite direction from the
empty-machine run: not "does the suite survive having nothing", but "does it survive having the
wrong thing". A fixture that only unsets on an already-clean machine would pass the third row and
fail the second.

## Turn 6 - publish, and what CI revealed

> publish ist freigegeben
> aktualisiere den issue gleich mit.

`tools/wikitool publish` first refused with `Missing option '--message'` (exit 2), then:

```
[main 31c9b81] test: autouse-Fixture härtet die Suite gegen stille Umgebungsabhängigkeiten (1.7.1, #8)
 8 files changed, 327 insertions(+), 3 deletions(-)
OK Published changes to origin/main.
```

Per `SOUL.md`, the tool's success line was not taken as proof. `git ls-remote` confirmed
`31c9b81` on `origin/main` and tag `v1.7.1` at the same commit.

CI run 79: green, all eight steps, **`702 passed in 15.84s`** - the fourth environment, and the
same number as the three local ones. Run 80 tagged the release. The `dist export` step reported
11 instructions and 5 skills against 14 and 6 in the dev tree, confirming that
`instructions/dev/testing-conventions.md` stays out of the distribution.

### The finding that retroactively justified the choice

Reading the run-79 log turned up something the issue could not have known:

```
Copying '/root/.gitconfig' to '/tmp/b93ea7a3-.../.gitconfig'
Temporarily overriding HOME='/tmp/b93ea7a3-...' before making global git config changes
[command]/usr/bin/git config --global --add safe.directory /workspace/torben/llm-wiki-test1
```

`actions/checkout@v7` now creates a global git configuration inside the container itself, and the
Tool-environment step writes `safe.directory` globally on top.

**The job container is therefore no longer reliably "the machine without a global
configuration".** It had that property in run 52 by accident. The rejected fallback - a second
hardened CI step - would have depended on that accident and would eventually have stopped
catching anything, silently, with no red run to say so. The fixture depends on none of it.

This was recorded in the CI comment on `.gitea/workflows/ci.yml`'s Tests step, so nobody adds the
second job back for the reason it was once needed.

## Turn 7 - the two open threads, filed rather than written down

Following `capture-session.md` step 4, both went to the tracker before the transcript:

- **#22** (`prio/3` `size/XS`) - `lint.py:157` counts `>` *lines*, not quote blocks. The
  `--fail-on-error` run during verification flagged
  `Source - Conversation - Auto Mode and Tool Choice Session 2026-08-31` for "4 quoted lines"
  when the page holds exactly one quote wrapped over four lines. The rule as written rewards
  overlong lines and punishes the repo's own wrap width. Not fixed in this session: different
  file, different rule, different change.
- **#23** (`prio/2` `size/S`) - nothing enforces that a newly introduced tool environment
  variable reaches `_WIKITOOL_ENV`. `testing-conventions.md` step 4 says to add it, but the
  premise of #8 was that a prose rule does not prevent this class of mistake. Noticed while
  writing that very step: the rule could be written down but not enforced.

## Outcome

| | |
|---|---|
| Version | `1.7.1` (PATCH - no command changes behaviour) |
| Commit | `31c9b81`, tag `v1.7.1` |
| Files | 8 changed, 327 insertions, 3 deletions |
| New | `tools/wiki_tools/tests/test_hermetic_env.py`, `instructions/dev/testing-conventions.md` |
| Changed | `conftest.py`, `test_new_page.py` (`git init -q -b main`), `.gitea/workflows/ci.yml`, `stack-dev/SKILL.md`, `CHANGES.md`, `VERSION` |
| Tests | 702 passed in four environments: developer shell, poisoned, `env -i`, CI container |
| Issues | #8 closed with two comments; #22 and #23 opened |
| CI | Run 79 green (all 8 steps), run 80 tagged the release |