Prerequisites
- A LiteLLM proxy that already routes model traffic.
- A Silmaril guardrail endpoint and API key from deployment access.
- If you self-host, use the guardrail endpoint from the same deployment as your classify endpoint.
SILMARIL_GUARDRAIL_URL and SILMARIL_API_KEY in the proxy environment.
Configure the guardrail
Start in warn mode while validating proxy coverage. This configuration omitsreview to inherit your deployment’s default review setting.
additional_provider_specific_params.on_error: warn controls how Silmaril
represents a valid would-block decision. It does not control LiteLLM’s handling
of guardrail endpoint failures. Keep LiteLLM’s fail_on_error default; setting
fail_on_error: false broadly bypasses guardrail errors.
Choose where the guardrail runs
default_on: true attaches this guardrail globally. To scope it instead, set
default_on: false and attach its guardrail_name in the model configuration
or in one request’s guardrails list. Do not define duplicate guardrails to
cover the same stage. For an unchanged response, one configured post_call
guardrail should run once even when the same name is attached at both model
and request level. If a native hook changes the response content, LiteLLM
checks the changed fingerprint again.
The repository qualification contract
contains reproducible commands for the qualified published
litellm[proxy]==1.108.3 CLI profile on Python 3.12. This profile validates
same-content deduplication, changed-content rechecks, buffered streaming, and
the failure policies described below. Verify your installed proxy version and
complete profile against that contract. Qualification establishes tested
compatibility; it does not establish deployment.
Expected result
LiteLLM calls Silmaril before and after the model call. In this starter configuration, a valid would-block decision is advisory and the proxy call continues.unreachable_fallback: fail_open is narrower than fail_on_error: false. It
allows unreachable transport failures and HTTP 502, 503, and 504
responses to continue. Malformed or non-object JSON and HTTP 500 remain
guardrail errors under LiteLLM’s default fail_on_error: true. A valid
Silmaril BLOCKED response is still a block when native enforcement is
explicitly configured.
Qualified endpoint and modality scope
The matrix below is bounded to the exact, separately hash-lockedlitellm[proxy]==1.108.3 package on Python 3.12. Qualification exercised the
published CLI and deterministic provider and Firewall services over loopback
only. It used no model, cloud service, paid inference endpoint, or customer
account. This establishes tested compatibility, not deployment or customer
activation.
Pass-through endpoints are not qualified by this matrix. Their visible fields
depend on endpoint/provider-specific JSONPath configuration, and LiteLLM
1.108.3 checks extracted pass-through text without applying response rewrites.
The endpoint rows are exercised directly rather than inferred from chat
coverage. Responses assertions preserve
instructions in both the exact
guardrail payload and provider request. Messages assertions preserve top-level
system, tool_result text, and client input tool_use through the actual
1.108.3 OpenAI Responses bridge. The provider returns the current Responses
function_call fields (id, call_id, name, arguments, and status);
qualification checks the exact guardrail tool-call payload and native Responses
or Anthropic tool_use delivery for clean calls, and verifies malicious
arguments are not delivered.
For buffered blocked streams, provider auditing records every emitted text
fragment and final source consumption. Endpoint-native SSE assertions prove
that no provider text fragment, including the safe prefix emitted before the
malicious fragment, reaches the client. Blocked paths also reject Responses
response.output_text.delta and Messages content_block_delta delivery.
The qualification also exercises LiteLLM’s endpoint-specific
structured_messages write-back for Responses and Messages. That case uses a
clearly marked synthetic GUARDRAIL_INTERVENED response only at the upstream
compatibility boundary. Production Silmaril responses remain NONE or
BLOCKED; this does not claim a Silmaril rewrite feature.
Native blocking and streaming
Native blocking is a separate, approved profile. Only the exact provider modemode: enforce selects it; do not add it to the starter configuration without
completing the agreed qualification. In that profile, a blocked pre-call can
stop provider execution, and a blocked non-streaming post-call can withhold
the response.
Streaming requires an additional delivery decision. Tokens already delivered
before a post-call verdict cannot be retracted, and
streaming_end_of_stream_only: true does not withhold them. If blocking must
prevent output exposure, qualify the complete proxy profile with
streaming_buffer_until_moderated: true and keep output buffered until the
verdict.
Optional review selection
Keepreview omitted to inherit your deployment’s default as shown above. Select standard or deep with additional_provider_specific_params.review.
mode: shadow explicitly selects Shadow response mode and takes precedence over the on_error: warn response-mode fallback. Review selection and response mode are independent.
standard selects standard review. deep requests a second opinion on eligible potentially harmful events and requires a deployment with deep review available. Otherwise Silmaril returns HTTP 400 before classification. An explicit deep selection overrides a standard-review default. Explicit null, wrong types, whitespace or case variants, and other strings are invalid.
See review depth for availability and response handling.
A successful explicit selection is acknowledged by the same exact string in top-level silmaril_review. This confirms the applied setting. An event may complete without additional review. Omitted review needs no acknowledgement and preserves the legacy response shape.