LiteLLM proxy

The AxioRank guardrail runs inside your LiteLLM proxy and governs three surfaces on every request, for every framework and language behind the proxy, with no per-agent code:

• Prompt (pre-call): the model input is scored before the model runs. A deny blocks the request (prompt injection, a leaked secret, a model or spend policy).
• Completion (post-call): the model output is scored. A redact policy swaps in a masked copy (secrets and PII); a deny blocks it.
• Tool calls (post-call): every tool call the model proposes is scored. Denied calls are stripped and replaced with a short, model-readable refusal, so the agent can re-plan.

Install

pip install "axiorank[litellm]"

Option 1: native guardrails config (recommended)

LiteLLM loads a custom guardrail from a Python file next to your config.yaml, so add a one-line re-export:

from axiorank.integrations.litellm import AxioRankGuardrail

Then register it. Two entries govern input and output, and default_on: true runs the guardrail on every request:

guardrails:
  - guardrail_name: "axiorank-input"
    litellm_params:
      guardrail: axiorank_guardrail.AxioRankGuardrail
      mode: "pre_call"
      default_on: true
  - guardrail_name: "axiorank-output"
    litellm_params:
      guardrail: axiorank_guardrail.AxioRankGuardrail
      mode: "post_call"
      default_on: true

Set your agent key in the environment (the file-based loader reads it from there):

export AXIORANK_API_KEY=axr_live_...
# optional, for a self-hosted deployment:
export AXIORANK_BASE_URL=https://your-axiorank.example.com

Option 2: callbacks (a single instance covers input and output)

from axiorank import AsyncAxioRank
from axiorank.integrations.litellm import axiorank_guardrail

proxy_handler_instance = axiorank_guardrail(
    AsyncAxioRank(api_key="axr_live_..."),
)

litellm_settings:
  callbacks: ["custom_callbacks.proxy_handler_instance"]

Turn on prompt and completion governance

Tool-call governance works out of the box. Prompt and completion content governance is gated per workspace, so it stays a monitor-only no-op until you switch it on:

1. In Settings, Workspace, find Model I/O guardrails and click Enable.
2. Add evaluation policies for the model surfaces: a deny policy on model.* (for prompt injection or a leaked secret) and a redact policy on model.completion (to mask secrets and PII in responses).

Until both are set, only tool calls are enforced.

What happens on deny

By default (on_deny="block"), a denied tool call is stripped from the response and replaced with a short refusal the agent can re-plan around, and a denied completion is replaced with a refusal. on_deny="raise" fails the whole request instead. A denied prompt always blocks the request before the model runs.

Next steps

• Framework integrations: per-agent adapters when you are not behind a proxy.
• Python SDK: the underlying client.