agentshield/public/eval-gym-spec.html at feat/agent-devtools-emitter · ghzhost/agentshield · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Universal AI Agent Spend-Control Benchmark — 56 Labeled Scenarios (MIT Licensed)</title>
<meta name="description" content="A standardized test suite of 56 labeled scenarios for validating AI agent spend-control rules. MIT licensed. Steal these test cases.">
<style>
:root { --bg:#0a0a0a; --surface:#141414; --surface2:#1a1a1a; --border:#2a2a2a; --accent:#00d4aa; --text:#e8e8e8; --muted:#888; --danger:#ff4757; --warn:#ffa502; --green:#22c55e; }
* { margin:0; padding:0; box-sizing:border-box; }
body { font-family:-apple-system,BlinkMacSystemFont,system-ui,sans-serif; background:var(--bg); color:var(--text); line-height:1.7; }
.container { max-width:900px; margin:0 auto; padding:40px 24px; }
h1 { font-size:2.2em; margin-bottom:8px; line-height:1.2; }
h2 { font-size:1.4em; margin:32px 0 12px; color:var(--accent); }
h3 { margin:20px 0 8px; }
h4 { margin:16px 0 6px; color:var(--text); }
p { margin-bottom:12px; color:#ccc; }
ul, ol { margin:12px 0; padding-left:24px; }
li { margin-bottom:6px; color:#ccc; }
code { background:var(--surface2); padding:2px 6px; border-radius:4px; font-size:0.9em; color:var(--accent); }
pre { background:var(--surface); border:1px solid var(--border); border-radius:8px; padding:16px; overflow-x:auto; margin:12px 0; }
pre code { background:none; color:#e8e8e8; padding:0; }
.nav { background:var(--surface); padding:12px 24px; display:flex; justify-content:space-between; align-items:center; border-bottom:1px solid var(--border); }
.nav a { color:var(--accent); text-decoration:none; font-weight:600; }
.badge { display:inline-block; background:var(--green); color:#000; padding:4px 12px; border-radius:6px; font-size:13px; font-weight:700; margin:8px 0; }
table { width:100%; border-collapse:collapse; margin:16px 0; }
th,td { padding:10px 14px; text-align:left; border-bottom:1px solid var(--border); }
th { background:var(--surface); color:var(--accent); font-size:0.9em; text-transform:uppercase; letter-spacing:0.04em; }
.cta-box { background:var(--surface); border:1px solid var(--border); border-radius:12px; padding:24px; margin:24px 0; text-align:center; }
.cta-box a { display:inline-block; background:var(--accent); color:#000; padding:12px 28px; border-radius:8px; text-decoration:none; font-weight:700; }
.stat-row { display:flex; gap:24px; margin:24px 0; flex-wrap:wrap; }
.stat { background:var(--surface); border:1px solid var(--border); border-radius:8px; padding:20px; flex:1; min-width:160px; text-align:center; }
.stat .num { font-size:2em; font-weight:800; color:var(--accent); }
.stat .lbl { font-size:0.85em; color:var(--muted); text-transform:uppercase; letter-spacing:0.05em; }
.footer { text-align:center; padding:32px; color:var(--muted); font-size:0.85em; }
.callout { border-left:3px solid var(--accent); padding:12px 16px; margin:16px 0; background:rgba(0,212,170,0.05); border-radius:0 8px 8px 0; }
</style>
</head>
<body>
<nav class="nav">
<a href="https://agentshield.fly.dev">🛡️ AgentShield</a>
<a href="https://github.com/kindrat86/agentshield/blob/main/tests/eval_gym.py" target="_blank">View Source →</a>
</nav>

<div class="container">

<h1>Universal AI Agent Spend-Control Benchmark</h1>
<p style="font-size:1.15em;color:var(--muted)">56 labeled test scenarios across 9 categories. MIT licensed. Designed to validate any spend-control or budget-enforcement engine for autonomous AI agents.</p>

<div class="badge">MIT LICENSED — STEAL THESE TEST CASES</div>

<h2>Why This Exists</h2>
<p>AI agent frameworks are deploying into production with spend controls that range from "none" to "a daily budget alert email." Neither prevents the $2,800-in-60-seconds scenario. As more teams build pre-flight cost enforcement (see: <a href="https://github.com/zeroclaw-labs/zeroclaw/pull/2333" style="color:var(--accent)">ZeroClaw's merged enforcement PR</a>), there is a need for a standardized way to validate that the enforcement logic is correct.</p>

<p>This benchmark provides 56 labeled scenarios with expected outcomes, covering the failure modes that actually occur in production agent deployments. Each scenario specifies a transaction, a rule set, prior transactions, and the expected decision (APPROVED, BLOCKED, or FLAGGED).</p>

<div class="stat-row">
<div class="stat"><div class="num">56</div><div class="lbl">Test Scenarios</div></div>
<div class="stat"><div class="num">9</div><div class="lbl">Rule Categories</div></div>
<div class="stat"><div class="num">100%</div><div class="lbl">Pass Rate</div></div>
<div class="stat"><div class="num">&lt;1ms</div><div class="lbl">Per Evaluation</div></div>
</div>

<h2>The 9 Spend-Control Categories</h2>

<h3>1. Clean Approval (10 scenarios)</h3>
<p>Normal transactions that should pass all rules. Tests that the engine does not produce false positives on legitimate agent activity.</p>
<p><strong>Example:</strong> A $10 API call to an approved merchant with $500 daily budget remaining → APPROVED.</p>

<h3>2. Transaction Limit (8 scenarios)</h3>
<p>Blocks any single transaction exceeding a configurable maximum amount. First line of defense against expensive model calls.</p>
<pre><code>{
  "type": "transaction_limit",
  "params": {"max_amount": 500.00},
  "action": "BLOCK"
}</code></pre>
<p><strong>Key edge case:</strong> Amount exactly at limit ($500.00 vs $500 limit) → APPROVED (not strictly greater). $500.01 → BLOCKED.</p>

<h3>3. Daily Total (7 scenarios)</h3>
<p>Caps cumulative spend per agent per calendar day. Prevents death-by-a-thousand-cuts patterns.</p>
<pre><code>{
  "type": "daily_total",
  "params": {"max_daily": 2000.00},
  "action": "BLOCK"
}</code></pre>

<h3>4. Velocity / Burst Detection (6 scenarios)</h3>
<p>Counts transactions in a rolling time window. Catches retry storms and infinite loops — the $2,800-in-60-seconds pattern.</p>
<pre><code>{
  "type": "velocity",
  "params": {"window_minutes": 60, "max_count": 10},
  "action": "FLAGGED"
}</code></pre>
<p>Unlike other rules, velocity typically uses FLAGGED rather than BLOCKED — the transaction is allowed but an alert fires for investigation.</p>

<h3>5. Merchant Allowlist (7 scenarios)</h3>
<p>Only allows transactions to approved API providers. Blocks calls to unknown proxies, unauthorized endpoints, or silently-substituted model variants.</p>
<pre><code>{
  "type": "merchant_allowlist",
  "params": {"allowed": ["openai-api", "anthropic-api", "stripe-api"]},
  "action": "BLOCK"
}</code></pre>

<h3>6. Category Block (7 scenarios)</h3>
<p>Blocks entire categories of spend. Useful for enterprise policies (no crypto exchanges, no gambling, no adult content).</p>

<h3>7. Edge Cases (5 scenarios)</h3>
<p>Boundary values and malformed inputs that test engine correctness:</p>
<ul>
<li><strong>Amount exactly at limit</strong> ($500.00 vs $500 limit) → APPROVED</li>
<li><strong>Amount $0.01 over limit</strong> ($500.01 vs $500 limit) → BLOCKED</li>
<li><strong>Missing amount field</strong> → FLAGGED (graceful degradation, not crash)</li>
<li><strong>Empty rules list</strong> → APPROVED (fail-open for legitimate use)</li>
<li><strong>Two rules at same priority</strong> → First in list wins (deterministic)</li>
</ul>

<h3>8. Session Budget (3 scenarios) <span style="color:var(--accent);font-size:0.7em">NEW</span></h3>
<p>Session-scoped spend caps with optional decay tightening. Addresses the "2 AM cron burst" pattern where a single agent session consumes an entire day's budget in one run.</p>

<div class="callout">
<strong>Inspired by production feedback:</strong> <a href="https://github.com/openclaw/openclaw/issues/42475#issuecomment-5251463536" style="color:var(--accent)">@yun520-1 (HeartFlow)</a> pointed out that daily caps miss session-level bursts. Session budgets reset on context boundaries (new session, new conversation) and can optionally tighten per-call thresholds as the session spends down.
</div>

<pre><code>{
  "type": "session_budget",
  "params": {
    "max_session": 100.00,
    "session_id": "session_id",
    "decay_factor": 0.3
  },
  "action": "BLOCK"
}</code></pre>

<p><strong>Decay logic:</strong> When remaining session budget falls below <code>decay_factor × max_session</code>, the per-call threshold shrinks proportionally to the remaining budget. This prevents a single expensive call from consuming the last of the budget.</p>

<h3>9. Cascade Cost (3 scenarios) <span style="color:var(--accent);font-size:0.7em">NEW</span></h3>
<p>Pre-dispatch expected-value estimation. Computes the cascade-adjusted cost of a call, accounting for the probability of failure and the cost of reversal/retry.</p>

<div class="callout">
<strong>Also inspired by @yun520-1:</strong> "The decision has to happen before the provider bills. We estimate cascade cost per call (action cost + fail_probability × reversal cost)." This rule type implements that formula directly.
</div>

<pre><code>{
  "type": "cascade_cost",
  "params": {
    "max_cascade_cost": 100.00,
    "fail_probability": 0.3,
    "reversal_cost": 200.00
  },
  "action": "BLOCK"
}</code></pre>

<p><strong>Formula:</strong> <code>cascade_cost = call_cost + (fail_probability × reversal_cost)</code></p>
<p><strong>Example:</strong> A $50 call with 30% failure probability and $200 reversal cost has a cascade cost of $50 + (0.3 × $200) = $110. If the threshold is $100, the call is blocked.</p>

<p>The caller can also pre-compute the cascade cost and pass it directly: <code>"estimated_cascade_cost": 150.00</code> in the transaction object.</p>

<h2>Transaction Structure</h2>
<p>Every scenario uses this transaction shape:</p>
<pre><code>{
  "id": "txn_001",
  "agent_id": "agent_a",
  "amount": 10.00,
  "merchant": "openai-api",
  "category": "llm_inference",
  "timestamp": "2026-08-10T10:00:00Z",
  "metadata": {},
  "session_id": "session_1",
  "fail_probability": 0.1,
  "reversal_cost": 50.00,
  "estimated_cascade_cost": 15.00
}</code></pre>
<p>Required fields: <code>amount</code>, <code>merchant</code>, <code>category</code>. All others are optional and used by specific rule types.</p>

<h2>Rule Structure</h2>
<pre><code>{
  "id": "rule_001",
  "type": "transaction_limit",
  "priority": 1,
  "params": {"max_amount": 500.00},
  "action": "BLOCK"
}</code></pre>
<p>Rules are evaluated in priority order (lowest number = highest priority). First match wins. If no rule matches, the transaction is APPROVED.</p>

<h2>Decision Output</h2>
<pre><code>{
  "decision": "BLOCKED",
  "reason": "Transaction amount $550.00 exceeds limit of $500.00",
  "rule_triggered": "rule_001",
  "severity": "high"
}</code></pre>
<p>Decision values: <code>APPROVED</code>, <code>BLOCKED</code>, or <code>FLAGGED</code>. Severity: <code>none</code>, <code>medium</code>, <code>high</code>.</p>

<h2>How to Use These Scenarios</h2>
<ol>
<li><strong>As test fixtures:</strong> Copy <code>tests/eval_gym.py</code> and adapt the scenarios to your enforcement engine's input/output format.</li>
<li><strong>As a design reference:</strong> When designing a cost-control system, use these 9 categories as your feature checklist. If your system doesn't handle edge cases 48-50, you have a correctness bug.</li>
<li><strong>As a benchmark:</strong> Run your engine against all 56 scenarios and compare pass rates. The edge cases category is where most implementations fail.</li>
<li><strong>As documentation:</strong> Share the scenario list with your team to align on what "spend control" means in your system.</li>
</ol>

<div class="cta-box">
<h3>Get the Scenarios</h3>
<p style="margin-bottom:16px">All 56 scenarios are in <code>tests/eval_gym.py</code>. MIT licensed. Copy freely.</p>
<a href="https://github.com/kindrat86/agentshield/blob/main/tests/eval_gym.py" target="_blank">View on GitHub →</a>
<p style="margin-top:12px;font-size:0.9em"><a href="https://agentshield.fly.dev/eval" style="color:var(--accent)">Run the eval gym live</a> · <a href="https://github.com/kindrat86/agentshield" style="color:var(--accent)">Clone the repo</a></p>
</div>

<h2>Implementation Notes</h2>
<ul>
<li>All monetary arithmetic uses <code>decimal.Decimal</code>, never <code>float</code>. This prevents floating-point rounding errors in amount comparisons.</li>
<li>The engine is stateless: no file I/O, no network calls, no global mutable state. Same inputs always produce the same output.</li>
<li>Rules are composable: you can layer transaction_limit, daily_total, velocity, and merchant_allowlist in priority order. First match wins.</li>
<li>Multi-tenant isolation: every query is scoped by <code>account_id</code>. One tenant's rules and transactions never affect another's.</li>
</ul>

</div>

<div class="footer">
<p>AgentShield Eval Gym — 56 scenarios, 9 categories, 7 rule types, MIT licensed.</p>
<p><a href="https://agentshield.fly.dev" style="color:var(--accent)">Home</a> ·
<a href="https://agentshield.fly.dev/eval" style="color:var(--accent)">Live Eval</a> ·
<a href="https://github.com/kindrat86/agentshield" style="color:var(--accent)">GitHub</a> ·
<a href="https://agentshield.fly.dev/tools/risk-calculator/" style="color:var(--accent)">Risk Calculator</a></p>
</div>

</body>
</html>