Coverage-Untergrenze 85 % in tools/.coveragerc, gegen beobachtete 87,0 % (4.7.2, schließt #10, eröffnet #51)
Files changed: - .gitea/workflows/ci.yml - CHANGES.md - EVALS.md - VERSION - tools/.coveragerc
This commit is contained in:
@@ -120,12 +120,12 @@ jobs:
|
|||||||
# container is no longer a special environment worth a second run.
|
# container is no longer a special environment worth a second run.
|
||||||
# See instructions/dev/testing-conventions.md.
|
# See instructions/dev/testing-conventions.md.
|
||||||
#
|
#
|
||||||
# Coverage is reported, not enforced: there is deliberately no
|
# Coverage is measured and enforced at a floor of 85% against a measured
|
||||||
# `--cov-fail-under` yet (Gitea #10). The threshold gets set in its own
|
# 87.0% - `fail_under` in tools/.coveragerc, not a flag here, so the
|
||||||
# later commit, with the measured number as its justification - one
|
# number sits next to the reasoning that produced it. It was set only
|
||||||
# picked before the number is either too low to bite or too high to
|
# after the number had been watched across 38 runs (Gitea #10, closed).
|
||||||
# survive the next honest commit, and the second kind gets lowered
|
# A red suite from this floor means coverage actually fell; the two
|
||||||
# instead of earned. Config: tools/.coveragerc.
|
# points of headroom already absorb a new thin Typer wrapper.
|
||||||
run: |
|
run: |
|
||||||
set -eu
|
set -eu
|
||||||
cd tools
|
cd tools
|
||||||
|
|||||||
+47
@@ -35,6 +35,53 @@ dev-checkout concern - readable here, never shipped as something to parse.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 4.7.2 - 2026-09-04 - Coverage-Untergrenze bei 85 %, gegen beobachtete 87,0 %
|
||||||
|
|
||||||
|
**Author:** Torben Nehmer
|
||||||
|
|
||||||
|
<!-- wikitool:bumps -->
|
||||||
|
- Coverage-Untergrenze 85 % in tools/.coveragerc
|
||||||
|
<!-- /wikitool:bumps -->
|
||||||
|
|
||||||
|
Die Suite hat jetzt einen Boden: `fail_under = 85` in `tools/.coveragerc`, gemessen gegen 87,0 %
|
||||||
|
(CI-Lauf 163, 6498 Statements, 975 Tests). Damit ist Gitea #10 geschlossen — das Issue, das die
|
||||||
|
Messung eingerichtet und die Schwelle danach **absichtlich** zurückgehalten hat, bis die Zahl
|
||||||
|
beobachtet war.
|
||||||
|
|
||||||
|
Die Beobachtung ist der eigentliche Inhalt dieses Bumps. Zwischen der ersten Messung (86,9 % von
|
||||||
|
5105 Statements, 730 Tests, Lauf 87, Stack 1.8.1) und heute ist der gemessene Code um ein Viertel
|
||||||
|
gewachsen und die Suite um ein Drittel, über 38 grüne Läufe — und die Quote hat sich um einen
|
||||||
|
Zehntelpunkt bewegt. Eine Untergrenze, die auf dieser Beobachtung steht, ist etwas anderes als
|
||||||
|
eine gegriffene Zahl.
|
||||||
|
|
||||||
|
**85 und nicht 87, und das ist keine Bequemlichkeit.** Der Coverage-Bericht unterscheidet drei
|
||||||
|
Sorten ungedeckter Zeilen, und nur eine davon bedeutet Arbeit (`EVALS.md` § „How much of the
|
||||||
|
stack the suite reaches"). Ein neuer dünner Typer-Wrapper senkt den Gesamtwert, ohne dass
|
||||||
|
irgendetwas schlechter geworden wäre — seine Logik liegt daneben und ist dort getestet. Eine
|
||||||
|
Schwelle auf dem gemessenen Wert würde genau an diesem Commit rot, und eine Schwelle, die aus
|
||||||
|
einem Nicht-Grund rot wird, wird gesenkt statt verdient. Das ist die Fehlerweise, die #10
|
||||||
|
verhindern wollte, nur von der anderen Seite. Die zwei Punkte sind der Platz, den die Taxonomie
|
||||||
|
verlangt.
|
||||||
|
|
||||||
|
`fail_under` steht in der Konfiguration und nicht als `--cov-fail-under` im CI-Schritt: so sitzt
|
||||||
|
die Zahl neben der Begründung, die sie erzeugt hat, und gilt für jeden `--cov`-Lauf statt nur für
|
||||||
|
den einen, den CI schreibt.
|
||||||
|
|
||||||
|
Was der Boden **nicht** tut: die drei echten Lücken schließen (`provenance_cmd.py` 44 %,
|
||||||
|
`migrate_cmd.py` 65 %, `type_resolver.py` 79 %). Er friert den erreichten Stand ein. Diese Liste
|
||||||
|
ist die einzige, die sich nicht bewegt hat, während alles um sie herum wuchs — `migrate_cmd.py`
|
||||||
|
ist sogar von 71 % gefallen, weil das Modul gewachsen ist und die neuen Zeilen ungetestet ankamen.
|
||||||
|
Das ist Gitea #51.
|
||||||
|
|
||||||
|
Mitgenommen, weil es dieselbe Frage beantwortet: der Coverage-Bericht **ist** als Artefakt
|
||||||
|
abrufbar, über die Run-Seite. Die Actions-Artefakt-Endpunkte melden dafür `total_count: 0`, weil
|
||||||
|
`upload-artifact@v3` über die ältere Artifact-API ablegt, die diese Endpunkte nicht lesen. Eine
|
||||||
|
leere Liste ist kein fehlgeschlagener Upload — steht jetzt in `EVALS.md` und im Kommentar an der
|
||||||
|
`Coverage report`-Stufe, damit die naheliegende „Korrektur" auf v4 (hier eingeschränkt) niemandem
|
||||||
|
mehr einfällt.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## 4.7.1 - 2026-09-04 - redundant_see_also in tools/CONTRACT.md und wiki-lint dokumentiert; xref-remove-Falle benannt
|
## 4.7.1 - 2026-09-04 - redundant_see_also in tools/CONTRACT.md und wiki-lint dokumentiert; xref-remove-Falle benannt
|
||||||
|
|
||||||
**Author:** Torben Nehmer
|
**Author:** Torben Nehmer
|
||||||
|
|||||||
@@ -249,15 +249,19 @@ created but not yet written reports broken links. That is the scaffold saying it
|
|||||||
|
|
||||||
### How much of the stack the suite reaches
|
### How much of the stack the suite reaches
|
||||||
|
|
||||||
Coverage is measured in CI and reported, never enforced - `pytest --cov`, config in
|
Coverage is measured in CI - `pytest --cov`, config in `tools/.coveragerc`, HTML and XML
|
||||||
`tools/.coveragerc`, HTML and XML uploaded as the `coverage-<run id>` artifact of every run.
|
uploaded as the `coverage-<run id>` artifact of every run.
|
||||||
**Fetch that artifact from the run's own page, not from the API**: `upload-artifact@v3` writes
|
**Fetch that artifact from the run's own page, not from the API**: `upload-artifact@v3` writes
|
||||||
through the older artifact API, and the Actions artifact REST endpoints answer `total_count: 0`
|
through the older artifact API, and the Actions artifact REST endpoints answer `total_count: 0`
|
||||||
for a run whose artifact the run page offers for download. The upload works; only the listing
|
for a run whose artifact the run page offers for download. The upload works; only the listing
|
||||||
does not see it. Do not re-derive this, and do not read the empty list as a failed upload.
|
does not see it. Do not re-derive this, and do not read the empty list as a failed upload.
|
||||||
|
|
||||||
There is no `--cov-fail-under`: a threshold is owed (Gitea #10), in its own commit, once the
|
It is enforced at a floor of **85%** (`fail_under` in `tools/.coveragerc`), which is what a red
|
||||||
number has been watched long enough to freeze the state it actually reached.
|
suite from this axis means: coverage actually fell, not that a wrapper was added. The floor was
|
||||||
|
set only after the number had been watched - Gitea #10 held it back for exactly that, and the
|
||||||
|
two points between 85 and the measured 87.0% are the room the taxonomy below asks for. A
|
||||||
|
threshold at the measured number goes red on the next thin Typer wrapper, and a threshold that
|
||||||
|
goes red for a non-reason gets lowered rather than earned.
|
||||||
|
|
||||||
**Measured 2026-09-04, stack 4.7.1: 87.0% of 6498 statements across `chemenu/`, 975 tests** -
|
**Measured 2026-09-04, stack 4.7.1: 87.0% of 6498 statements across `chemenu/`, 975 tests** -
|
||||||
CI run 163. The first measurement, at stack 1.8.1 on 2026-08-31, was 86.9% of 5105 statements
|
CI run 163. The first measurement, at stack 1.8.1 on 2026-08-31, was 86.9% of 5105 statements
|
||||||
@@ -284,10 +288,10 @@ low, and three kinds have to be told apart before any of it turns into work:
|
|||||||
`fetcher` parameter for exactly that, and the real network line stays uncovered on purpose.
|
`fetcher` parameter for exactly that, and the real network line stays uncovered on purpose.
|
||||||
- **Genuine gaps**, where uncovered lines are logic nobody exercises: `provenance_cmd.py`
|
- **Genuine gaps**, where uncovered lines are logic nobody exercises: `provenance_cmd.py`
|
||||||
(44%), `migrate_cmd.py` (65%), `type_resolver.py` (79%). This is the list worth reading, and
|
(44%), `migrate_cmd.py` (65%), `type_resolver.py` (79%). This is the list worth reading, and
|
||||||
the reason step 2 of #10 is not a formality. It is also the list that has not moved while
|
the only one of the three that has not moved while everything around it did:
|
||||||
everything around it did: `provenance_cmd.py` sits where it sat, and `migrate_cmd.py` fell
|
`provenance_cmd.py` sits where it sat, and `migrate_cmd.py` fell from 71% because it grew and
|
||||||
from 71% because it grew and its new lines arrived untested. A threshold freezes this; it does
|
its new lines arrived untested. The floor freezes this; it does not close it. Closing it is
|
||||||
not close it.
|
Gitea #51.
|
||||||
|
|
||||||
## Scoring a session
|
## Scoring a session
|
||||||
|
|
||||||
|
|||||||
+22
-4
@@ -11,10 +11,27 @@
|
|||||||
# measure the stack's tests - so an addopts entry would break the plain
|
# measure the stack's tests - so an addopts entry would break the plain
|
||||||
# `pytest -q` that every local run and the CI "Tests" step use.
|
# `pytest -q` that every local run and the CI "Tests" step use.
|
||||||
#
|
#
|
||||||
# No `fail_under` yet, on purpose: Gitea #10 sets the threshold in a separate,
|
# `fail_under` lives here rather than as a `--cov-fail-under` flag in the CI
|
||||||
# later commit, once the measured number exists to justify it. A threshold
|
# step, so the number sits next to the reasoning that produced it and applies to
|
||||||
# picked before the number is either too low to bite or too high to survive the
|
# any `--cov` run, not just the one CI happens to write.
|
||||||
# next honest commit - and the second kind gets lowered rather than earned.
|
#
|
||||||
|
# 85, against a measured 87.0% (CI run 163, 6498 statements, 975 tests). Gitea
|
||||||
|
# #10 held this back until the number had been watched: the first measurement
|
||||||
|
# was 86.9% of 5105 statements over 730 tests (CI run 87), and between the two
|
||||||
|
# the measured code grew by a quarter and the suite by a third while the quota
|
||||||
|
# moved a tenth of a point. That stability is what the threshold rests on.
|
||||||
|
#
|
||||||
|
# The two points of headroom are not slack. They are the room the report's own
|
||||||
|
# taxonomy asks for: a new thin Typer wrapper lowers the total without anything
|
||||||
|
# having got worse, because its logic is tested beside it (see EVALS.md § "How
|
||||||
|
# much of the stack the suite reaches"). A threshold at the measured number
|
||||||
|
# would go red on exactly that commit, and a threshold that goes red for a
|
||||||
|
# non-reason gets lowered rather than earned - which is the failure mode #10
|
||||||
|
# existed to avoid, arriving from the other side.
|
||||||
|
#
|
||||||
|
# What this number does *not* do is close the genuine gaps - provenance_cmd.py,
|
||||||
|
# migrate_cmd.py, type_resolver.py. It freezes the state that was reached; the
|
||||||
|
# gaps are their own work, tracked separately.
|
||||||
[run]
|
[run]
|
||||||
source = chemenu
|
source = chemenu
|
||||||
omit =
|
omit =
|
||||||
@@ -23,3 +40,4 @@ omit =
|
|||||||
[report]
|
[report]
|
||||||
show_missing = True
|
show_missing = True
|
||||||
precision = 1
|
precision = 1
|
||||||
|
fail_under = 85
|
||||||
|
|||||||
Reference in New Issue
Block a user