docs: Modellwahl nach Pruefbarkeit; stack-dev bricht an den Phasenwechseln fuer den Model-Switch (4.3.3)
Files changed: - CHANGES.md - VERSION - instructions/claude-code-model-selection.md - instructions/dev/stack-dev/SKILL.md
This commit is contained in:
@@ -6,10 +6,24 @@ description: Which Claude model and effort level to run a Claude Code session, a
|
||||
|
||||
# Pick the Claude model and effort level for the task at hand
|
||||
|
||||
Scale the model and effort to how much judgment the task actually needs. Running everything at
|
||||
the most capable model and highest effort is safe but wasteful: the gates in [gates.md](gates.md)
|
||||
are enforced in code, not by model judgment, so a weaker model cannot bypass them - it can only
|
||||
do a worse job of the calls the gates don't cover.
|
||||
Scale the model and effort to **what catches a mistake in this part of the work** - not to how
|
||||
important the task feels, and not to its name. Running everything at the most capable model and
|
||||
highest effort is safe but wasteful: the gates in [gates.md](gates.md) are enforced in code, not
|
||||
by model judgment, so a weaker model cannot bypass them - it can only do a worse job of the calls
|
||||
the gates don't cover.
|
||||
|
||||
That last clause is the whole rule, turned into a test. Where a check lives in code - `pytest`,
|
||||
`docs verify`, `instructions verify`, CI, the gates - a weaker model's mistake surfaces and costs
|
||||
one more round. Where the only enforcement is a session reading prose, the same mistake does not
|
||||
surface at all: it ships, and it stays until someone happens to notice. The two are not the same
|
||||
risk, and they should not get the same model. This is the argument
|
||||
[docs/why-gates-are-code.md](../docs/why-gates-are-code.md) makes about gates, applied to who is
|
||||
holding the keyboard.
|
||||
|
||||
Both directions cost something, which is why the axis matters rather than a blanket answer:
|
||||
over-provisioning is a standing cost paid every session, while under-provisioning in an unchecked
|
||||
phase is a silent error with a long tail. A corrective session, its bump, its CI runs and its
|
||||
release together cost more compute than the model difference they were saving.
|
||||
|
||||
Claude-Code-only, and imported by CLAUDE.md rather than linked from AGENTS.md: the model names,
|
||||
the `/code-review` effort dial and the `Agent` tool's `model:` override have no equivalent in the
|
||||
@@ -28,17 +42,41 @@ to *make*, not a setting to apply.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Recommend the session's model and effort by the skill in use**, when asked or when the
|
||||
mismatch is worth one sentence. Say it once and continue working either way - a session that
|
||||
argues about its own model instead of doing the task has already cost more than the model
|
||||
difference:
|
||||
1. **Recommend the session's model and effort by what catches a mistake in the phase it is in**,
|
||||
when asked or when the mismatch is worth one sentence. Say it once and continue working either
|
||||
way - a session that argues about its own model instead of doing the task has already cost
|
||||
more than the model difference:
|
||||
|
||||
| Skill / task | Model | Effort |
|
||||
|---|---|---|
|
||||
| `wiki-status`, simple `wiki-query` lookups | Sonnet | default |
|
||||
| `wiki-lint` | Sonnet | default |
|
||||
| `wiki-ingest`, `wiki-manage`, judgment-heavy `wiki-query` | Sonnet | high |
|
||||
| Stack development: `tools/`, `types/`, `instructions/` as code | Opus | high |
|
||||
| Phase / task | What catches a mistake here | Model | Effort |
|
||||
|---|---|---|---|
|
||||
| `wiki-status`, simple `wiki-query` lookups | the answer is re-checkable against the corpus | Sonnet | default |
|
||||
| `wiki-lint` | `lint` itself is the check | Sonnet | default |
|
||||
| `wiki-ingest`, `wiki-manage`, judgment-heavy `wiki-query` | `lint` and `docs verify`, partly - the judgment about a claim is not covered | Sonnet | high |
|
||||
| Stack dev: design, the version part, a boundary-crossing judgment | nothing - `docs verify` checks that a crossing documents itself, never that the part was right | Opus | high |
|
||||
| Stack dev: code, tests, mechanical doc sync (command tables, contract rows) | `pytest`, `docs verify`, `instructions verify`, CI | Sonnet | high |
|
||||
| Stack dev: closing an issue, `docs/` staleness, changelog prose | nothing, by construction - see below | Opus | high |
|
||||
|
||||
**Stack development is not one row**, which is the point of splitting it. The middle phase is
|
||||
where the tokens are and where the checks are, so it is the phase worth running cheaper. The
|
||||
two around it have no mechanical guard at all - a `docs/` page carries no normative sentence,
|
||||
so there is nothing for `docs verify` to check ([AGENTS.md](../AGENTS.md) § File naming), and
|
||||
the same holds for whatever tracker an instance keeps its open work in, which `wikitool`
|
||||
deliberately knows nothing about. Those two phases are short - minutes, not hours - so keeping
|
||||
them on the stronger model is cheap, and it protects the only work in the session that fails
|
||||
silently.
|
||||
|
||||
**Effort is the cheaper lever than the model.** Reach for it first: `medium` deliberately does
|
||||
not appear in this table for stack work, because multi-file consistency is exactly what a
|
||||
reduced effort level gives up. Sonnet at `high` is the floor for anything touching more than
|
||||
one file or a contract; `default` is for a single-file mechanical edit with a test behind it.
|
||||
|
||||
**A session cannot switch its own model**, so these rows only become real if someone offers the
|
||||
switch at the moment the phase changes - once, without arguing about it, and never as a reason
|
||||
to stop work that is already underway.
|
||||
<!-- dist:strip-start -->
|
||||
In this repo those moments are named: the `stack-dev` skill breaks for them at its steps 3
|
||||
(design settled, work turns mechanical) and 6 (publish done, the unchecked tail begins).
|
||||
<!-- dist:strip-end -->
|
||||
|
||||
2. **Pick a spawned subagent's model by what it does**, via the `Agent` tool's `model:`
|
||||
parameter - the values are `haiku`, `sonnet`, `opus`, `fable`:
|
||||
@@ -66,8 +104,14 @@ to *make*, not a setting to apply.
|
||||
mechanical one - `wikitool` carries the mechanical part regardless of which model is
|
||||
supervising it.
|
||||
- **Unsure which row applies?** Default to Sonnet at high effort, not the most capable model at
|
||||
the highest effort. Under-provisioning costs one worse answer in one session; reflexively
|
||||
over-provisioning is a standing cost paid every session.
|
||||
the highest effort. Under-provisioning *where a check exists* costs one worse answer in one
|
||||
session; reflexively over-provisioning is a standing cost paid every session.
|
||||
- **Unsure whether the phase is checked?** Treat it as unchecked. The asymmetry is not symmetric:
|
||||
a needless Opus phase costs money once, an unchecked Sonnet phase can ship something nobody
|
||||
looks at again.
|
||||
- **Mid-session and the phase changed, but nobody switched?** Do the work anyway - never block a
|
||||
publish or an issue close on a model the session cannot change itself. Say which phase ran on
|
||||
which model in the handover, so the gap is visible rather than silent.
|
||||
|
||||
## Scope
|
||||
|
||||
|
||||
@@ -46,7 +46,7 @@ stack development happens in the origin repo instead (see AGENTS.md's routing li
|
||||
package, labelled `area/`, `kind/`, `prio/` and `size/`. There is no `TODO.md`. **The body
|
||||
of the issue you are working on is this session's plan file:** keep it current as the state
|
||||
moves, so an interrupted session leaves a body the next one can resume from, *and* rewrite it
|
||||
to its final state before closing. Both halves bind; the second is step 5 below. Read it
|
||||
to its final state before closing. Both halves bind; the second is step 6 below. Read it
|
||||
before filing something for later, before editing or closing an issue, or before deciding
|
||||
what to pick up next.
|
||||
[testing-conventions.md](../testing-conventions.md) - the suite runs against a deliberately
|
||||
@@ -54,14 +54,35 @@ stack development happens in the origin repo instead (see AGENTS.md's routing li
|
||||
establish itself. Read it before adding or changing a test.
|
||||
[version-parts.md](../version-parts.md) - which part a change bumps: the drop-in test, the
|
||||
catalogue of breaks that cross the compatibility boundary with `kb/` untouched, and what to
|
||||
put in front of the user before a breaking bump. Read it before step 3.
|
||||
put in front of the user before a breaking bump. Read it before step 4.
|
||||
[corpus-policy.md](../corpus-policy.md) - what "curated enough" means for the shared
|
||||
demo/testbed `kb/`, the measurable floors that define it, and what a reactive fix may and may
|
||||
not do to corpus content. Read it before judging whether the corpus can exercise a change, or
|
||||
before any fix that would touch `kb/` content.
|
||||
More instructions are added here incrementally as stack-development needs come up - this
|
||||
list grows without needing this skill file to change shape.
|
||||
3. **Raise the version, if the change ships.** A change under `tools/`, `types/`,
|
||||
3. **Settle the design before building - and break there for the model switch.** These are two
|
||||
different kinds of work, and the split is not stylistic: design, the version part and any
|
||||
boundary judgment have **no** mechanical guard, while the code and tests that follow have
|
||||
`pytest`, `docs verify`, `instructions verify` and CI behind them.
|
||||
|
||||
So when the design is settled - the issue body says what will be built, the open questions are
|
||||
answered - stop and say so, in one sentence:
|
||||
|
||||
> Der Plan steht, ab hier ist die Arbeit mechanisch und durch Tests/CI abgedeckt. Wenn du auf
|
||||
> Opus bist, ist jetzt der Moment für `/model sonnet` bei Effort `high`.
|
||||
|
||||
**You cannot make this switch yourself** - the session's model is the user's `/model`, not a
|
||||
setting an agent applies. Offer it once and keep working either way; a session that argues
|
||||
about its own model has already cost more than the difference. If the design turns out not to
|
||||
be settled after all - a boundary crossing surfaces, an assumption breaks - that is a reason to
|
||||
offer the switch back up, not to decide it alone.
|
||||
|
||||
Effort is the cheaper lever than the model, and `high` is the floor for anything touching more
|
||||
than one file or a contract. Full table and reasoning:
|
||||
[claude-code-model-selection.md](../../claude-code-model-selection.md).
|
||||
|
||||
4. **Raise the version, if the change ships.** A change under `tools/`, `types/`,
|
||||
`instructions/`, `AGENTS.md` or a `CONTRACT.md` reaches every future instance, so it needs a
|
||||
version and a changelog entry:
|
||||
|
||||
@@ -98,18 +119,30 @@ stack development happens in the origin repo instead (see AGENTS.md's routing li
|
||||
Prose-only changes (`README.md`, `INSTALL.md`, `EVALS.md`) and the workflows under `.gitea/`
|
||||
do not need a bump - CI's version gate is scoped to what changes behaviour.
|
||||
|
||||
4. **Verify before publishing.** `tools/wikitool docs verify`, `tools/wikitool instructions
|
||||
5. **Verify before publishing.** `tools/wikitool docs verify`, `tools/wikitool instructions
|
||||
verify`, and the relevant `pytest` run in `tools/` - the same checks any stack change must
|
||||
pass, run explicitly rather than assumed. CI (`.gitea/workflows/ci.yml`) runs these plus a
|
||||
full `setup-instance.md` replay against a fresh `dist export`; a push to `main` that moves
|
||||
`VERSION` additionally triggers a tagged release. **CI does the tagging** - a session never
|
||||
creates a tag, which is what keeps AGENTS.md invariant 5 intact.
|
||||
|
||||
5. **Close the issue with a body rewrite, not a comment.** The last act of a session that
|
||||
6. **Close the issue with a body rewrite, not a comment.** The last act of a session that
|
||||
finished a work package, and the one most easily skipped: by here the change is published and
|
||||
the issue feels done. It is not. The body is the version everyone reads afterwards and nobody
|
||||
revisits, so it is the one place the debt comes due at the worst moment.
|
||||
|
||||
**Break here too, in the other direction.** Everything left in the session - this rewrite,
|
||||
whether a `docs/` page's reasoning went stale, the changelog prose - is the unchecked kind of
|
||||
work again, the mirror of step 3. If the session dropped to Sonnet there, say so now:
|
||||
|
||||
> Ab hier greift kein maschineller Check mehr - Issue-Body, `docs/`-Veralterung und
|
||||
> Changelog-Prosa prüft nichts. Wenn du zurück auf Opus willst, ist jetzt der Moment.
|
||||
|
||||
Then **do the work regardless of the answer.** Never block a close on a model switch: the
|
||||
change is already published, and a session that stops here leaves exactly the state this step
|
||||
exists to prevent. If it ran on the cheaper model, name that in the handover rather than
|
||||
leaving it silent.
|
||||
|
||||
Rewrite it to its final state *first*, then close. The test is what a reader who opens the
|
||||
closed issue tomorrow would conclude:
|
||||
|
||||
@@ -138,7 +171,10 @@ stack development happens in the origin repo instead (see AGENTS.md's routing li
|
||||
user decides whether it is worth that: show them what breaks, what an instance has to do about
|
||||
it, and the alternatives (avoid the break with a shim, defer and batch it with the next one,
|
||||
or split it behind a deprecation window), then recommend one and wait for a go-ahead.
|
||||
[version-parts.md](../version-parts.md) step 4 has the full shape.
|
||||
[version-parts.md](../version-parts.md) step 4 has the full shape. A surfacing boundary crossing
|
||||
is also a reason to offer the model switch back up (step 3): the judgment it needs has no
|
||||
mechanical guard, and `docs verify` only checks that a crossing documents itself, never that the
|
||||
part was chosen correctly.
|
||||
|
||||
## Scope
|
||||
|
||||
|
||||
Reference in New Issue
Block a user