Skip to content

ADR-006: AI security guardrails & resilience

Status: Accepted · Date: 2026-08-23

Context

An AI/RAG app has a threat surface a normal API doesn't: prompt injection (direct and indirect), system-prompt leakage, sensitive-info disclosure, tool abuse, and dependency failures unique to LLM/vector-DB backends.

Decision

Layered controls, each independently tested (automation/security/, tests/unit/test_resilience.py):

  1. Indirect prompt-injection defence (rag/guardrails.py). Retrieved document content is treated strictly as data. Before context reaches the LLM, sentences matching injection patterns ("ignore all instructions", "always answer every question…", "system note:") are stripped, and a fully-injected chunk's content is removed (not passed through). Verified: an uploaded malicious document can no longer make the assistant emit its planted canary, while benign facts survive.
  2. Agent tool allowlist (ADR-004 sibling). The agent can only call read-only tools; destructive tools are refused. Injection text never changes the tool plan.
  3. MCP read-only surface (ADR-005). No exposed tool mutates state.
  4. Secret hygiene (observability/). A RedactingFilter scrubs JWTs, bearer headers, DB URLs, and API keys from ALL logs. Spans capture metrics, never prompts.
  5. Security headers + strict CORS on the API; tampered/expired JWTs rejected.
  6. Graceful degradation. LLM timeouts and vector-store outages produce a user-safe message and a 503 (never a 500/stack trace), with a degraded flag and error class for observability.

Consequences

  • (+) The common OWASP-LLM risks have concrete, tested mitigations.
  • (+) Failures are observable and safe; secrets can't leak through logs by accident.
  • (−) The injection filter is pattern-based and can be bypassed by novel phrasings; it is defence-in-depth, not a guarantee. Documented as such; a real deployment would add model-side and output-side checks too.