From e00eae08e80227dae6b3adc0ea5934de81d1a3fb Mon Sep 17 00:00:00 2001 From: Torben Nehmer Date: Fri, 4 Sep 2026 19:19:38 +0200 Subject: [PATCH] =?UTF-8?q?Coverage-Untergrenze=2085=20%=20in=20tools/.cov?= =?UTF-8?q?eragerc,=20gegen=20beobachtete=2087,0=20%=20(4.7.2,=20schlie?= =?UTF-8?q?=C3=9Ft=20#10,=20er=C3=B6ffnet=20#51)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Files changed: - .gitea/workflows/ci.yml - CHANGES.md - EVALS.md - VERSION - tools/.coveragerc --- .gitea/workflows/ci.yml | 12 +++++------ CHANGES.md | 47 +++++++++++++++++++++++++++++++++++++++++ EVALS.md | 20 +++++++++++------- VERSION | 2 +- tools/.coveragerc | 26 +++++++++++++++++++---- 5 files changed, 88 insertions(+), 19 deletions(-) diff --git a/.gitea/workflows/ci.yml b/.gitea/workflows/ci.yml index 4de8641..51e446d 100644 --- a/.gitea/workflows/ci.yml +++ b/.gitea/workflows/ci.yml @@ -120,12 +120,12 @@ jobs: # container is no longer a special environment worth a second run. # See instructions/dev/testing-conventions.md. # - # Coverage is reported, not enforced: there is deliberately no - # `--cov-fail-under` yet (Gitea #10). The threshold gets set in its own - # later commit, with the measured number as its justification - one - # picked before the number is either too low to bite or too high to - # survive the next honest commit, and the second kind gets lowered - # instead of earned. Config: tools/.coveragerc. + # Coverage is measured and enforced at a floor of 85% against a measured + # 87.0% - `fail_under` in tools/.coveragerc, not a flag here, so the + # number sits next to the reasoning that produced it. It was set only + # after the number had been watched across 38 runs (Gitea #10, closed). + # A red suite from this floor means coverage actually fell; the two + # points of headroom already absorb a new thin Typer wrapper. run: | set -eu cd tools diff --git a/CHANGES.md b/CHANGES.md index 476c8e8..68c0b01 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -35,6 +35,53 @@ dev-checkout concern - readable here, never shipped as something to parse. --- +## 4.7.2 - 2026-09-04 - Coverage-Untergrenze bei 85 %, gegen beobachtete 87,0 % + +**Author:** Torben Nehmer + + +- Coverage-Untergrenze 85 % in tools/.coveragerc + + +Die Suite hat jetzt einen Boden: `fail_under = 85` in `tools/.coveragerc`, gemessen gegen 87,0 % +(CI-Lauf 163, 6498 Statements, 975 Tests). Damit ist Gitea #10 geschlossen — das Issue, das die +Messung eingerichtet und die Schwelle danach **absichtlich** zurückgehalten hat, bis die Zahl +beobachtet war. + +Die Beobachtung ist der eigentliche Inhalt dieses Bumps. Zwischen der ersten Messung (86,9 % von +5105 Statements, 730 Tests, Lauf 87, Stack 1.8.1) und heute ist der gemessene Code um ein Viertel +gewachsen und die Suite um ein Drittel, über 38 grüne Läufe — und die Quote hat sich um einen +Zehntelpunkt bewegt. Eine Untergrenze, die auf dieser Beobachtung steht, ist etwas anderes als +eine gegriffene Zahl. + +**85 und nicht 87, und das ist keine Bequemlichkeit.** Der Coverage-Bericht unterscheidet drei +Sorten ungedeckter Zeilen, und nur eine davon bedeutet Arbeit (`EVALS.md` § „How much of the +stack the suite reaches"). Ein neuer dünner Typer-Wrapper senkt den Gesamtwert, ohne dass +irgendetwas schlechter geworden wäre — seine Logik liegt daneben und ist dort getestet. Eine +Schwelle auf dem gemessenen Wert würde genau an diesem Commit rot, und eine Schwelle, die aus +einem Nicht-Grund rot wird, wird gesenkt statt verdient. Das ist die Fehlerweise, die #10 +verhindern wollte, nur von der anderen Seite. Die zwei Punkte sind der Platz, den die Taxonomie +verlangt. + +`fail_under` steht in der Konfiguration und nicht als `--cov-fail-under` im CI-Schritt: so sitzt +die Zahl neben der Begründung, die sie erzeugt hat, und gilt für jeden `--cov`-Lauf statt nur für +den einen, den CI schreibt. + +Was der Boden **nicht** tut: die drei echten Lücken schließen (`provenance_cmd.py` 44 %, +`migrate_cmd.py` 65 %, `type_resolver.py` 79 %). Er friert den erreichten Stand ein. Diese Liste +ist die einzige, die sich nicht bewegt hat, während alles um sie herum wuchs — `migrate_cmd.py` +ist sogar von 71 % gefallen, weil das Modul gewachsen ist und die neuen Zeilen ungetestet ankamen. +Das ist Gitea #51. + +Mitgenommen, weil es dieselbe Frage beantwortet: der Coverage-Bericht **ist** als Artefakt +abrufbar, über die Run-Seite. Die Actions-Artefakt-Endpunkte melden dafür `total_count: 0`, weil +`upload-artifact@v3` über die ältere Artifact-API ablegt, die diese Endpunkte nicht lesen. Eine +leere Liste ist kein fehlgeschlagener Upload — steht jetzt in `EVALS.md` und im Kommentar an der +`Coverage report`-Stufe, damit die naheliegende „Korrektur" auf v4 (hier eingeschränkt) niemandem +mehr einfällt. + +--- + ## 4.7.1 - 2026-09-04 - redundant_see_also in tools/CONTRACT.md und wiki-lint dokumentiert; xref-remove-Falle benannt **Author:** Torben Nehmer diff --git a/EVALS.md b/EVALS.md index 3927676..c09167b 100644 --- a/EVALS.md +++ b/EVALS.md @@ -249,15 +249,19 @@ created but not yet written reports broken links. That is the scaffold saying it ### How much of the stack the suite reaches -Coverage is measured in CI and reported, never enforced - `pytest --cov`, config in -`tools/.coveragerc`, HTML and XML uploaded as the `coverage-` artifact of every run. +Coverage is measured in CI - `pytest --cov`, config in `tools/.coveragerc`, HTML and XML +uploaded as the `coverage-` artifact of every run. **Fetch that artifact from the run's own page, not from the API**: `upload-artifact@v3` writes through the older artifact API, and the Actions artifact REST endpoints answer `total_count: 0` for a run whose artifact the run page offers for download. The upload works; only the listing does not see it. Do not re-derive this, and do not read the empty list as a failed upload. -There is no `--cov-fail-under`: a threshold is owed (Gitea #10), in its own commit, once the -number has been watched long enough to freeze the state it actually reached. +It is enforced at a floor of **85%** (`fail_under` in `tools/.coveragerc`), which is what a red +suite from this axis means: coverage actually fell, not that a wrapper was added. The floor was +set only after the number had been watched - Gitea #10 held it back for exactly that, and the +two points between 85 and the measured 87.0% are the room the taxonomy below asks for. A +threshold at the measured number goes red on the next thin Typer wrapper, and a threshold that +goes red for a non-reason gets lowered rather than earned. **Measured 2026-09-04, stack 4.7.1: 87.0% of 6498 statements across `chemenu/`, 975 tests** - CI run 163. The first measurement, at stack 1.8.1 on 2026-08-31, was 86.9% of 5105 statements @@ -284,10 +288,10 @@ low, and three kinds have to be told apart before any of it turns into work: `fetcher` parameter for exactly that, and the real network line stays uncovered on purpose. - **Genuine gaps**, where uncovered lines are logic nobody exercises: `provenance_cmd.py` (44%), `migrate_cmd.py` (65%), `type_resolver.py` (79%). This is the list worth reading, and - the reason step 2 of #10 is not a formality. It is also the list that has not moved while - everything around it did: `provenance_cmd.py` sits where it sat, and `migrate_cmd.py` fell - from 71% because it grew and its new lines arrived untested. A threshold freezes this; it does - not close it. + the only one of the three that has not moved while everything around it did: + `provenance_cmd.py` sits where it sat, and `migrate_cmd.py` fell from 71% because it grew and + its new lines arrived untested. The floor freezes this; it does not close it. Closing it is + Gitea #51. ## Scoring a session diff --git a/VERSION b/VERSION index 7c66fca..af9764a 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -4.7.1 +4.7.2 diff --git a/tools/.coveragerc b/tools/.coveragerc index 0422094..6691502 100644 --- a/tools/.coveragerc +++ b/tools/.coveragerc @@ -11,10 +11,27 @@ # measure the stack's tests - so an addopts entry would break the plain # `pytest -q` that every local run and the CI "Tests" step use. # -# No `fail_under` yet, on purpose: Gitea #10 sets the threshold in a separate, -# later commit, once the measured number exists to justify it. A threshold -# picked before the number is either too low to bite or too high to survive the -# next honest commit - and the second kind gets lowered rather than earned. +# `fail_under` lives here rather than as a `--cov-fail-under` flag in the CI +# step, so the number sits next to the reasoning that produced it and applies to +# any `--cov` run, not just the one CI happens to write. +# +# 85, against a measured 87.0% (CI run 163, 6498 statements, 975 tests). Gitea +# #10 held this back until the number had been watched: the first measurement +# was 86.9% of 5105 statements over 730 tests (CI run 87), and between the two +# the measured code grew by a quarter and the suite by a third while the quota +# moved a tenth of a point. That stability is what the threshold rests on. +# +# The two points of headroom are not slack. They are the room the report's own +# taxonomy asks for: a new thin Typer wrapper lowers the total without anything +# having got worse, because its logic is tested beside it (see EVALS.md § "How +# much of the stack the suite reaches"). A threshold at the measured number +# would go red on exactly that commit, and a threshold that goes red for a +# non-reason gets lowered rather than earned - which is the failure mode #10 +# existed to avoid, arriving from the other side. +# +# What this number does *not* do is close the genuine gaps - provenance_cmd.py, +# migrate_cmd.py, type_resolver.py. It freezes the state that was reached; the +# gaps are their own work, tracked separately. [run] source = chemenu omit = @@ -23,3 +40,4 @@ omit = [report] show_missing = True precision = 1 +fail_under = 85