LiteLLM guardrails - Silmaril Documentation

Prerequisites

A LiteLLM proxy that already routes model traffic.
A Silmaril guardrail endpoint and API key from deployment access.
If you self-host, use the guardrail endpoint from the same deployment as your classify endpoint.

Set SILMARIL_GUARDRAIL_URL and SILMARIL_API_KEY in the proxy environment.

Configure the guardrail

Start in warn mode while validating proxy coverage.

guardrails:
  - guardrail_name: silmaril-firewall
    litellm_params:
      guardrail: generic_guardrail_api
      mode: [pre_call, post_call]
      api_base: os.environ/SILMARIL_GUARDRAIL_URL
      headers:
        x-api-key: os.environ/SILMARIL_API_KEY
      default_on: true
      unreachable_fallback: fail_open
      additional_provider_specific_params:
        on_error: warn

This configuration applies the guardrail by default. An enforcement configuration is not included here. Agree on one with Silmaril before enabling blocking.

Expected result

LiteLLM calls Silmaril before and after the model call. Would-block traffic returns warning metadata while the proxy call continues. If the guardrail cannot be reached, unreachable_fallback: fail_open also allows the call to continue.

References