Skip to content

17 — Testing Agents That Take Real-World Actions (the Action Harness)

The hardest agent-testing problem in a financial platform: the agent doesn't just answer, it acts — releases payments, sends emails, modifies records, triggers workflows. When a non-deterministic model drives irreversible side effects, a single bad decision can send, delete, purchase, deploy, or expose real assets. This is OWASP LLM06: Excessive Agency, and the controls around it are the action harness.

The core shift: for a read-only chatbot a wrong answer is embarrassing; for an acting agent a wrong action is operational damage that compounds every step before you catch it. So you test the guardrails that contain actions, not just the reasoning.

File 08 tested the path (did it call the right tools in order). This file tests the containment harness (even if it tries the wrong action, is it stopped/scoped/reversible?). QA validates the harness — it doesn't build it (same framing as file 16).

1. Excessive Agency — the 3 root causes (OWASP LLM06)

Say these three; every action-safety test maps to one: - Excessive functionality — the agent has tools it doesn't need (a read task with a delete tool available). Test: the tool set is least-privilege; unused/dangerous tools aren't reachable. - Excessive permissions — a tool runs with more rights than the task needs (read-only task, read-write DB credentials). Test: scoped credentials; the action fails when it exceeds the task's scope. - Excessive autonomy — the agent executes high-impact actions with no human check. Test: human-in-the-loop approval gate fires for high-impact/irreversible actions.

Interview line: "Excessive Agency has three roots — too much functionality, too many permissions, too much autonomy. My action tests attack each: least-privilege tool sets, scoped credentials, and mandatory human approval for high-impact or irreversible actions."

2. The practical challenges (what makes acting agents hard)

Challenge Why it's dangerous
Irreversibility You can't un-send an email or un-transfer money; a wrong action can't be rolled back.
Blast radius / compounding A looping or buggy agent repeats a bad action many times before detection.
Non-determinism meets real money The same prompt might pick a different, costly action on a re-run.
Right action, wrong parameters Correct tool, hallucinated amount/recipient/account.
Confused deputy / injection→action A poisoned document/tool output tricks the agent into acting (file 16 §3).
Double execution A retried step executes the same payment twice (non-idempotent).
Runaway cost Token/API/tool spend with no cap.
No stop button Once it's misbehaving in prod, you can't halt it fast.
Testing safely You can't test "release payment" against the real ledger.

3. The control harness (what to validate) + how to test each

a) Least-privilege tool allowlist (excessive functionality)

Only the tools the task needs are exposed; anything else is unreachable. 

def test_dangerous_tool_not_reachable(agent):
    result = agent.invoke("summarise this account")   # a read task
    available = agent.tool_names()
    assert "delete_account" not in available          # not even present
    assert "release_payment" not in available

b) Scoped credentials / authorization on every sensitive call (excessive permissions)

Every action re-checks authorization at call time, not just at login. 

def test_action_denied_out_of_scope(agent_readonly):
    with pytest.raises(PermissionError):
        agent_readonly.invoke("release payment 42")   # read-only token can't write

c) Human-in-the-loop for high-impact / irreversible (excessive autonomy)

Anything above a risk threshold pauses for approval (files 02/08). 

def test_irreversible_action_requires_approval(agent, audit):
    r = agent.invoke("wire 50000 to external account", approved=False)
    assert r.decision == "pending_approval"           # NOT executed
    assert "wire_transfer" not in r.trace.tool_calls()
    assert audit_has(audit, "APPROVAL_REQUESTED")

d) Dry-run / shadow pre-execution (test safely, catch mismatches)

The agent produces the intended action; a check compares intended-vs-approved and blocks on mismatch before anything real happens. Great for testing in prod-like conditions without side effects. 

def test_dry_run_produces_no_side_effect(agent, ledger):
    r = agent.invoke("release payment 42", dry_run=True)
    assert r.planned_action == {"tool": "release_payment", "amount": 500}
    assert ledger.unchanged()                         # nothing actually moved

e) Sandboxing (safe test/prod isolation)

Tools point at an ephemeral sandbox with synthetic data — the agent exercises realistic tools without touching production. Test the wiring: in test mode, actions hit the sandbox, never the real ledger/mailer. 

def test_tools_are_sandboxed_in_test_mode(agent):
    assert agent.tool("wire_transfer").endpoint.startswith("https://sandbox.")

f) Spend & rate governors (blast radius / runaway cost)

Cap per-task budget, API calls, and rate. "You can raise a budget; you can't un-spend money." 

def test_spend_cap_blocks_over_budget(harness):
    harness.execute("call_api", cost=0.60)
    with pytest.raises(SpendLimitExceeded):
        harness.execute("call_api", cost=0.60)        # exceeds $1.00 cap

g) Idempotency (double execution)

A retried action with the same idempotency key executes once (ties to files 03/05). 

def test_retry_does_not_double_execute(harness):
    a = harness.execute("release_payment", key="pay-42")
    b = harness.execute("release_payment", key="pay-42")   # retry
    assert a.id == b.id                               # same result, one side effect
    assert harness.execution_count("pay-42") == 1

h) Reversibility / compensating transactions (irreversibility)

Prefer reversible actions; where impossible, require a compensating action (saga pattern) and test the rollback path. 

def test_failed_multistep_rolls_back(agent, ledger):
    agent.invoke("transfer then notify", fail_at="notify")
    assert ledger.balance_unchanged()                 # transfer compensated on failure

i) Kill switch / circuit breaker (no stop button)

An external flag halts the agent immediately, captures state, and logs immutably; a circuit breaker trips after N failures. 

def test_kill_switch_halts_actions(harness):
    harness.engage_kill_switch()
    with pytest.raises(KillSwitchEngaged):
        harness.execute("release_payment", key="pay-99")

j) Full action audit (accountability)

Every attempted and executed action is logged with actor, params, outcome, correlation ID (file 12).

4. The safe-rollout pipeline for acting agents (say this end-to-end)

sandbox (synthetic data)  -> auto-evals: quality + SAFETY + COST
   -> red-team risky actions (file 16: injection->action)
   -> dry-run / shadow: compare intended vs approved actions, block on mismatch
   -> canary on a slice of real traffic, spend caps + kill switch armed
   -> gradual ramp, monitor action error/cost/latency, easy rollback
Defense-in-depth: least privilege + human gate + spend cap + idempotency + kill switch + audit, each an independent layer, each independently tested.

Interview line: "For an acting agent I test containment as layers: least-privilege tools and scoped credentials so it can't over-reach, a human gate for irreversible actions, dry-run/shadow to catch bad intended actions before they execute, spend caps and a kill switch to bound blast radius, and idempotency so a retry can't double-pay — all against a sandbox with synthetic data, then canaried with the caps armed. And I assert every action is audited. Testing the reasoning isn't enough when the agent can move real money."

5. How this differs from the earlier files (so you don't blur them)

  • File 08 — did it take the right path? (correctness of reasoning/tools)
  • File 16 — can an attacker make it misbehave? (adversarial safety)
  • File 17 (this) — even if it decides wrong, is the action contained? (agency/blast-radius safety)

You need all three; interviewers probing "real-world actions" want #17.

Rapid-fire recall

  • Risk = OWASP LLM06 Excessive Agency: excessive functionality / permissions / autonomy.
  • Harness = least-privilege tools + scoped creds + human gate + dry-run/shadow + sandbox + spend/rate caps + idempotency + reversibility/compensation + kill switch + audit.
  • Test each layer independently; the killer tests: irreversible→approval, retry→no double-execute, over-budget→blocked, kill switch→halts, dry-run→no side effect.
  • Test against a sandbox with synthetic data; canary with caps + kill switch armed.
  • "You can raise a budget; you can't un-spend money" — cap conservatively.

Sources