fix: budget gate and loop-breaker refusals exit without a traceback (Gitea #147)
Files changed: - CHANGES.md - VERSION - tools/chemenu/cli.py - tools/chemenu/commands/run_budget.py - tools/chemenu/tests/test_run_budget.py
This commit is contained in:
1 parent
8f61209b6b
commit
bc314e5c8c
5 files changed
+132
-16
No files matched your search
+19
-1
@@ -59,7 +59,7 @@ concern - readable here, never shipped as something to parse.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 7.1.0-beta.26 - 2026-09-26 - docs contract: the merged-stream test pins its own ON FAILURE line
|
## 7.1.0-beta.27 - 2026-09-26 - Budget gate and loop-breaker refusals exit without a traceback
|
||||||
|
|
||||||
**Author:** Torben Nehmer
|
**Author:** Torben Nehmer
|
||||||
|
|
||||||
@@ -71,6 +71,7 @@ concern - readable here, never shipped as something to parse.
|
|||||||
- CalDAV task-tracker provider (Nextcloud Tasks, iOS Reminders); review reports unknown-value findings instead of skipping them
|
- CalDAV task-tracker provider (Nextcloud Tasks, iOS Reminders); review reports unknown-value findings instead of skipping them
|
||||||
- dist export no longer cuts the dist export record out of the shipped tools/CONTRACT.md
|
- dist export no longer cuts the dist export record out of the shipped tools/CONTRACT.md
|
||||||
- fail() prints the command's ON FAILURE lines on stderr
|
- fail() prints the command's ON FAILURE lines on stderr
|
||||||
|
- Budget gate and loop-breaker refusals exit without a traceback
|
||||||
|
|
||||||
**Low impact**
|
**Low impact**
|
||||||
- version bump no longer points at version release in its output
|
- version bump no longer points at version release in its output
|
||||||
@@ -97,6 +98,23 @@ concern - readable here, never shipped as something to parse.
|
|||||||
- docs contract: the merged-stream test pins its own ON FAILURE line
|
- docs contract: the merged-stream test pins its own ON FAILURE line
|
||||||
<!-- /wikitool:bumps -->
|
<!-- /wikitool:bumps -->
|
||||||
|
|
||||||
|
### Budget gate and loop-breaker refusals exit without a traceback
|
||||||
|
|
||||||
|
`cli.main()` calls `run_budget.record_and_check()` before Typer ever dispatches to a subcommand,
|
||||||
|
so a refusal leaves through `_util.fail()`'s `typer.Exit` outside any Click context - nothing
|
||||||
|
caught it there, so the process exited 1 correctly but printed a Python traceback right after the
|
||||||
|
`ERROR` line (Gitea #147). That traceback sat exactly in the output AGENTS.md § Gates asks a
|
||||||
|
session to show a human and stop on; it reads as a crash rather than a gate and invites the retry
|
||||||
|
the message forbids (Gitea #52). `main()` now catches `typer.Exit` around that one call and exits
|
||||||
|
with its code directly, and two new subprocess tests - through the real `python -m chemenu.cli`
|
||||||
|
entry point, not the direct `record_and_check()` call the existing cross-process tests use -
|
||||||
|
pin that a tripped gate leaves neither stream carrying a traceback, still emits `gate.refused`,
|
||||||
|
and neither records a `wikitool.call` nor advances the session's counter for the refused call
|
||||||
|
itself. Both refusal messages, the `budget` command group's own help text, and one code comment
|
||||||
|
also pointed at "the tooling contract's 'Iteration and Cost Limits'" or "...Tool Error
|
||||||
|
Contracts" - neither section exists by that name - and now name AGENTS.md § Tool error contract
|
||||||
|
and `instructions/gates.md` § Iteration Budget Gate and loop-breaker, the sections that do.
|
||||||
|
|
||||||
### CalDAV task-tracker provider (Nextcloud Tasks, iOS Reminders); review reports unknown-value findings instead of skipping them
|
### CalDAV task-tracker provider (Nextcloud Tasks, iOS Reminders); review reports unknown-value findings instead of skipping them
|
||||||
|
|
||||||
A second task-tracker adapter, `caldav` (RFC 4791/5545), so an instance is not bound to Super
|
A second task-tracker adapter, `caldav` (RFC 4791/5545), so an instance is not bound to Super
|
||||||
|
|||||||
+19
-7
@@ -266,13 +266,13 @@ _typer_click_core.Command.format_help = _contract_format_help
|
|||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
def main() -> None:
|
||||||
# Iteration Budget Gate / Loop-Breaker (see the tooling contract's
|
# Iteration Budget Gate / Loop-Breaker (see instructions/gates.md
|
||||||
# "Iteration and Cost Limits"): recorded and enforced here, once per
|
# "Iteration Budget Gate and loop-breaker"): recorded and enforced here,
|
||||||
# process, before Typer dispatches to any subcommand - so it covers every
|
# once per process, before Typer dispatches to any subcommand - so it
|
||||||
# command uniformly and cannot be bypassed by the calling agent skipping a
|
# covers every command uniformly and cannot be bypassed by the calling
|
||||||
# step. Help output is never counted: discovering a command's options is
|
# agent skipping a step. Help output is never counted: discovering a
|
||||||
# not iteration on the wiki, and charging for it would discourage exactly
|
# command's options is not iteration on the wiki, and charging for it
|
||||||
# the behavior the skills ask for.
|
# would discourage exactly the behavior the skills ask for.
|
||||||
#
|
#
|
||||||
# Tracing sits at the same point for the same reason - one place that no
|
# Tracing sits at the same point for the same reason - one place that no
|
||||||
# command can route around. It is not the same set, though: the budget
|
# command can route around. It is not the same set, though: the budget
|
||||||
@@ -284,7 +284,19 @@ def main() -> None:
|
|||||||
override = "--override-budget" in argv
|
override = "--override-budget" in argv
|
||||||
filtered = [a for a in argv if a != "--override-budget"]
|
filtered = [a for a in argv if a != "--override-budget"]
|
||||||
command = filtered[0] if filtered else ""
|
command = filtered[0] if filtered else ""
|
||||||
|
# A gate refusal leaves `record_and_check` through `_util.fail()`,
|
||||||
|
# which raises `typer.Exit` (Gitea #147). That is fine inside Typer's
|
||||||
|
# own dispatch - Click catches it - but this call runs *before*
|
||||||
|
# `app()` ever starts, so nothing catches it here: left alone, the
|
||||||
|
# process would exit 1 correctly but print a Python traceback right
|
||||||
|
# after the `ERROR` line, which AGENTS.md § Gates asks a session to
|
||||||
|
# show a human and stop on. A traceback reads as a crash, not a gate,
|
||||||
|
# and invites exactly the retry the message forbids (see #52).
|
||||||
|
try:
|
||||||
charged = run_budget.record_and_check(command, filtered[1:], override)
|
charged = run_budget.record_and_check(command, filtered[1:], override)
|
||||||
|
except typer.Exit as exc:
|
||||||
|
code = exc.exit_code
|
||||||
|
sys.exit(code if isinstance(code, int) else (0 if code is None else 1))
|
||||||
sys.argv = [sys.argv[0], *filtered]
|
sys.argv = [sys.argv[0], *filtered]
|
||||||
_run_traced(command, filtered[1:], charged)
|
_run_traced(command, filtered[1:], charged)
|
||||||
return
|
return
|
||||||
|
|||||||
@@ -33,7 +33,7 @@ from chemenu.session import session_id as _shared_session_id
|
|||||||
from chemenu.session import session_id_source as _shared_session_id_source
|
from chemenu.session import session_id_source as _shared_session_id_source
|
||||||
from chemenu.telemetry import emit
|
from chemenu.telemetry import emit
|
||||||
|
|
||||||
app = typer.Typer(help="Session iteration/cost budget gate (see the tooling contract's 'Iteration and Cost Limits').")
|
app = typer.Typer(help="Session iteration/cost budget gate (see instructions/gates.md 'Iteration Budget Gate and loop-breaker').")
|
||||||
|
|
||||||
STATE_DIR = config.ROOT / "tools" / ".wikitool_session"
|
STATE_DIR = config.ROOT / "tools" / ".wikitool_session"
|
||||||
STATE_FILE = STATE_DIR / "budget.json"
|
STATE_FILE = STATE_DIR / "budget.json"
|
||||||
@@ -206,8 +206,8 @@ def loop_breaker_message(call_signature: str, loop_window: int) -> str:
|
|||||||
return (
|
return (
|
||||||
f"Loop-Breaker: the last {loop_window} wikitool calls in this session were "
|
f"Loop-Breaker: the last {loop_window} wikitool calls in this session were "
|
||||||
f"identical ('{call_signature}'). This usually means the agent is stuck retrying "
|
f"identical ('{call_signature}'). This usually means the agent is stuck retrying "
|
||||||
"the same failing operation instead of changing approach - per the tooling contract's "
|
"the same failing operation instead of changing approach - per AGENTS.md's "
|
||||||
"Tool Error Contracts, that is exactly the case for stopping and escalating rather than "
|
"Tool error contract, that is exactly the case for stopping and escalating rather than "
|
||||||
"retrying again. Stop, explain the situation to the user, and get explicit direction "
|
"retrying again. Stop, explain the situation to the user, and get explicit direction "
|
||||||
"before continuing. Only re-run with --override-budget once the user has confirmed "
|
"before continuing. Only re-run with --override-budget once the user has confirmed "
|
||||||
"the repeat is intentional - never add it on the agent's own initiative."
|
"the repeat is intentional - never add it on the agent's own initiative."
|
||||||
@@ -217,7 +217,8 @@ def loop_breaker_message(call_signature: str, loop_window: int) -> str:
|
|||||||
def call_limit_message(count: int, call_limit: int) -> str:
|
def call_limit_message(count: int, call_limit: int) -> str:
|
||||||
return (
|
return (
|
||||||
f"Iteration Budget Gate: this session has made {count} wikitool calls, exceeding the "
|
f"Iteration Budget Gate: this session has made {count} wikitool calls, exceeding the "
|
||||||
f"limit of {call_limit}. Per the tooling contract's 'Iteration and Cost Limits' section, a "
|
f"limit of {call_limit}. Per instructions/gates.md's 'Iteration Budget Gate and "
|
||||||
|
"loop-breaker' section, a "
|
||||||
"single task should typically need roughly 5-15 calls (simple) or 20-35 (complex multi-tool "
|
"single task should typically need roughly 5-15 calls (simple) or 20-35 (complex multi-tool "
|
||||||
"workflow like wiki-ingest/wiki-lint). This far past that band is a documented sign of poor "
|
"workflow like wiki-ingest/wiki-lint). This far past that band is a documented sign of poor "
|
||||||
"task decomposition or a stuck loop. Stop, summarize progress and the blocker to the "
|
"task decomposition or a stuck loop. Stop, summarize progress and the blocker to the "
|
||||||
@@ -304,8 +305,8 @@ def refund() -> None:
|
|||||||
"""Give the current session its last charged slot back.
|
"""Give the current session its last charged slot back.
|
||||||
|
|
||||||
Called when the command declined instead of acting: a rejected argument,
|
Called when the command declined instead of acting: a rejected argument,
|
||||||
or a read-only check reporting findings (`_util.fail`, exit 1). The
|
or a read-only check reporting findings (`_util.fail`, exit 1). AGENTS.md's
|
||||||
tooling contract answers a rejected argument with "fix it and retry once",
|
Tool error contract answers a rejected argument with "fix it and retry once",
|
||||||
so charging for the rejection makes the prescribed response cost two slots
|
so charging for the rejection makes the prescribed response cost two slots
|
||||||
for one operation - and the budget exists to bound iteration on the wiki,
|
for one operation - and the budget exists to bound iteration on the wiki,
|
||||||
which a call that changed nothing did not do.
|
which a call that changed nothing did not do.
|
||||||
|
|||||||
@@ -47,11 +47,23 @@ def test_call_limit_trips_past_threshold():
|
|||||||
|
|
||||||
def test_call_limit_message_mentions_contract_and_override():
|
def test_call_limit_message_mentions_contract_and_override():
|
||||||
message = run_budget.call_limit_message(61, 60)
|
message = run_budget.call_limit_message(61, 60)
|
||||||
assert "Iteration and Cost Limits" in message
|
assert "Iteration Budget Gate and loop-breaker" in message
|
||||||
assert "--override-budget" in message
|
assert "--override-budget" in message
|
||||||
assert "61" in message and "60" in message
|
assert "61" in message and "60" in message
|
||||||
|
|
||||||
|
|
||||||
|
def test_message_sections_exist_in_the_documents_they_cite():
|
||||||
|
"""Both refusal messages point a reader at a section by name (Gitea
|
||||||
|
#147: they used to point at "the tooling contract's 'Iteration and Cost
|
||||||
|
Limits'" and "...Tool Error Contracts", neither of which exists). Pin
|
||||||
|
that the cited headings are real, so the two drift apart again only if
|
||||||
|
this test is touched too."""
|
||||||
|
agents_md = (config._PACKAGE_ROOT / "AGENTS.md").read_text(encoding="utf-8")
|
||||||
|
gates_md = (config._PACKAGE_ROOT / "instructions" / "gates.md").read_text(encoding="utf-8")
|
||||||
|
assert "## Tool error contract" in agents_md
|
||||||
|
assert "## Iteration Budget Gate and loop-breaker" in gates_md
|
||||||
|
|
||||||
|
|
||||||
def test_record_and_check_reports_whether_it_charged():
|
def test_record_and_check_reports_whether_it_charged():
|
||||||
assert run_budget.record_and_check("new", ["entity"], override=False) is True
|
assert run_budget.record_and_check("new", ["entity"], override=False) is True
|
||||||
assert run_budget.record_and_check("search", ["anything"], override=False) is False
|
assert run_budget.record_and_check("search", ["anything"], override=False) is False
|
||||||
@@ -348,3 +360,76 @@ def test_the_loop_breaker_trips_across_separate_shells(tmp_path, monkeypatch):
|
|||||||
result = _spawn_call(tmp_path, monkeypatch, command="xref", args=args)
|
result = _spawn_call(tmp_path, monkeypatch, command="xref", args=args)
|
||||||
assert result.returncode != 0
|
assert result.returncode != 0
|
||||||
assert "Loop-Breaker" in result.stdout
|
assert "Loop-Breaker" in result.stdout
|
||||||
|
|
||||||
|
|
||||||
|
# --- a refusal exits without a traceback (Gitea #147) ---
|
||||||
|
#
|
||||||
|
# The two tests above call `run_budget.record_and_check` directly, so they
|
||||||
|
# never exercise `cli.main()` - the place `_util.fail()`'s `typer.Exit` used
|
||||||
|
# to go uncaught, since it is raised *before* `app()` starts and nothing
|
||||||
|
# outside `app()` catches it. These drive the real entry point
|
||||||
|
# (`python -m chemenu.cli`, not a `-c` snippet) so the whole `main()` codepath
|
||||||
|
# runs, and check the property the fix promises: exit 1, no traceback on
|
||||||
|
# either stream, `gate.refused` recorded, no `wikitool.call` for the refused
|
||||||
|
# invocation, and the budget counter left exactly where it was.
|
||||||
|
|
||||||
|
def _spawn_cli_call(tmp_path, trace_dir, monkeypatch, *, session_id, args):
|
||||||
|
monkeypatch.setenv("CHEMENU_ROOT", str(tmp_path))
|
||||||
|
monkeypatch.setenv("WIKI_TRACE_DIR", str(trace_dir))
|
||||||
|
monkeypatch.setenv("WIKITOOL_SESSION_ID", session_id)
|
||||||
|
monkeypatch.delenv("CLAUDE_CODE_SESSION_ID", raising=False)
|
||||||
|
return subprocess.run(
|
||||||
|
[sys.executable, "-m", "chemenu.cli", *args],
|
||||||
|
cwd=config._PACKAGE_ROOT / "tools",
|
||||||
|
capture_output=True, text=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _trace_events(trace_dir, session_id):
|
||||||
|
path = trace_dir / session_id / "trace.jsonl"
|
||||||
|
return [json.loads(line) for line in path.read_text(encoding="utf-8").splitlines() if line]
|
||||||
|
|
||||||
|
|
||||||
|
def test_loop_breaker_refusal_has_no_traceback(tmp_path, monkeypatch):
|
||||||
|
session_id = "test-147-loop"
|
||||||
|
trace_dir = tmp_path / "trace"
|
||||||
|
args = ["xref", "add", "--a", "X", "--b", "Y"]
|
||||||
|
for _ in range(run_budget.DEFAULT_LOOP_WINDOW):
|
||||||
|
_spawn_cli_call(tmp_path, trace_dir, monkeypatch, session_id=session_id, args=args)
|
||||||
|
|
||||||
|
result = _spawn_cli_call(tmp_path, trace_dir, monkeypatch, session_id=session_id, args=args)
|
||||||
|
assert result.returncode == 1
|
||||||
|
assert result.stdout.startswith("ERROR Loop-Breaker")
|
||||||
|
assert "Traceback" not in result.stdout
|
||||||
|
assert "Traceback" not in result.stderr
|
||||||
|
|
||||||
|
events = _trace_events(trace_dir, session_id)
|
||||||
|
assert [e["event"] for e in events[-1:]] == ["gate.refused"]
|
||||||
|
assert events[-1]["attrs"]["gate"] == "loop-breaker"
|
||||||
|
assert sum(e["event"] == "wikitool.call" for e in events) == run_budget.DEFAULT_LOOP_WINDOW
|
||||||
|
|
||||||
|
state = json.loads((tmp_path / "tools" / ".wikitool_session" / "budget.json").read_text())
|
||||||
|
assert state[session_id]["count"] == run_budget.DEFAULT_LOOP_WINDOW
|
||||||
|
|
||||||
|
|
||||||
|
def test_iteration_budget_refusal_has_no_traceback(tmp_path, monkeypatch):
|
||||||
|
session_id = "test-147-limit"
|
||||||
|
trace_dir = tmp_path / "trace"
|
||||||
|
for i in range(run_budget.DEFAULT_CALL_LIMIT):
|
||||||
|
_spawn_cli_call(tmp_path, trace_dir, monkeypatch, session_id=session_id, args=["lint", f"--pass-{i}"])
|
||||||
|
|
||||||
|
result = _spawn_cli_call(
|
||||||
|
tmp_path, trace_dir, monkeypatch, session_id=session_id, args=["lint", "--one-too-many"]
|
||||||
|
)
|
||||||
|
assert result.returncode == 1
|
||||||
|
assert result.stdout.startswith("ERROR Iteration Budget Gate")
|
||||||
|
assert "Traceback" not in result.stdout
|
||||||
|
assert "Traceback" not in result.stderr
|
||||||
|
|
||||||
|
events = _trace_events(trace_dir, session_id)
|
||||||
|
assert [e["event"] for e in events[-1:]] == ["gate.refused"]
|
||||||
|
assert events[-1]["attrs"]["gate"] == "iteration-budget"
|
||||||
|
assert sum(e["event"] == "wikitool.call" for e in events) == run_budget.DEFAULT_CALL_LIMIT
|
||||||
|
|
||||||
|
state = json.loads((tmp_path / "tools" / ".wikitool_session" / "budget.json").read_text())
|
||||||
|
assert state[session_id]["count"] == run_budget.DEFAULT_CALL_LIMIT
|
||||||
Reference in new issue
Block a user