Guardrails

Guardrails are the checks placed around a model to constrain what reaches the user or the rest of the system: filtering unacceptable content, validating that output matches a required format, refusing requests outside the intended scope, capping how much a process may spend or how many times it may retry.

They exist because a model is a probabilistic component in an otherwise deterministic system. You cannot make its output guaranteed; you can make the consequences of bad output bounded, which is what everything downstream needs in order to be dependable.

The most valuable guardrail is usually the least sophisticated: validating output against a strict schema and rejecting anything that does not conform, rather than hoping the instructions were followed.