AI Architecture Pattern 03: Structured Output And Validation

The routing queue does a literal enum match against three labels: open, under_review, resolved. The classification returned status: "still being looked at" — present, a string, exactly what Pattern 02’s contract required. It doesn’t match any label. The dispute sat unrouted with no error raised.

Root cause: Pattern 02’s contract has no mechanism to express “this string must be one of these exact values.” It validates presence and Python type only.

Fix: Pattern 03 adds an enum constraint on the status field via Bedrock Structured Outputs (outputConfig.textFormat.structure.jsonSchema). The schema prevents the model from generating any value outside the allowed set — constrained at generation time, not caught after.

Input validation (Pattern 02): checks data coming into the prompt — are required fields present? Are they the right Python types? Runs before the model call.

Output schema validation (Pattern 03): checks the model’s response — is it the expected structure? Are enum-constrained fields within the allowed set? Runs after the model call, or with Bedrock Structured Outputs, is enforced during generation.

Both are necessary: input validation protects the model from bad data (cheap, fast rejection). Output schema validation protects downstream systems from a model that produced a valid-looking but unusable value.

Post-hoc validation: model generates freely, a validation function checks the result. If wrong, reject and optionally retry. Problems: the bad value exists long enough to be logged, each retry costs another model call.

Constrained decoding (Bedrock Structured Outputs): the schema is compiled into a grammar. Each generation step is restricted to tokens that remain valid given the grammar. An enum-violating value is physically impossible to generate — no retry needed.

Use constrained decoding when the schema is within Bedrock’s supported subset (basic types, enum, required). Use post-hoc Pydantic for schema features outside that subset (numeric ranges, recursive schemas).

Supported: basic types (string, number, integer, boolean, array, object), enum on string fields, required fields list, some format values.

Not supported: numeric minimum/maximum, string minLength/maxLength, recursive schemas, full JSON Schema Draft 2020-12.

The hybrid pattern: Bedrock Structured Outputs for the enum constraint (strongest guarantee, zero retry cost) + Pydantic post-hoc for range checks that Bedrock’s subset can’t express (e.g. confidence in [0.0, 1.0]).

With Bedrock Structured Outputs (real AWS): the schema declares confidence as type: "number". Constrained decoding prevents the model from generating the string "high" in that position — only tokens that parse as a JSON number are valid.

Post-hoc Pydantic (belt-and-suspenders or fallback):

class DisputeClassification(BaseModel):
    status: Literal["open", "under_review", "resolved"]
    confidence: float = Field(ge=0.0, le=1.0)

isinstance("high", float) is False — caught before touching the routing queue.

Best practice: both. Bedrock Structured Outputs at generation; Pydantic on the Lambda side for range constraints Bedrock can’t express.

response = client.converse(
    modelId="amazon.nova-micro-v1:0",
    messages=[{"role": "user", "content": [{"text": prompt}]}],
    outputConfig={
        "textFormat": {
            "structure": {
                "jsonSchema": {
                    "schema": {
                        "type": "object",
                        "properties": {
                            "dispute_reference": {"type": "string"},
                            "status": {
                                "type": "string",
                                "enum": ["open", "under_review", "resolved"]
                            },
                            "confidence": {"type": "number"}
                        },
                        "required": ["dispute_reference", "status", "confidence"]
                    }
                }
            }
        }
    }
)

Bedrock compiles the schema into a grammar on the first call (~few hundred ms) and caches it for 24 hours. The response is a JSON string in response["output"]["message"]["content"][0]["text"]. Parse with json.loads().

Three problems:

1. JSON validity ≠ schema validity. json.loads() confirms the response is parseable JSON. It does not check field presence, types, or enum values. {"status": "still being looked at"} is valid JSON.

2. No field-level check. The routing queue does a literal enum match. An invalid status goes straight to the queue — no error raised, dispute unrouted.

3. Unstructured error response. {"error": "model did not return valid JSON"} gives no field name, no allowed values, no way to fix it fast.

Fix: add a Pydantic model with Literal on status — or use Bedrock Structured Outputs so the invalid value can’t be generated at all.

Choose Pydantic when:

• Numeric range constraints — e.g. confidence in [0.0, 1.0]. Bedrock has no minimum/maximum support.
• Recursive schemas — Bedrock doesn’t support self-referencing schemas.
• Not on Bedrock — Structured Outputs is a Bedrock feature; Pydantic works with any model output.
• Richer error messages — Pydantic ValidationError gives field-level detail including which value violated which constraint.

In practice, use both: Bedrock Structured Outputs for enum and type constraints (strongest guarantee); Pydantic as a belt-and-suspenders check for constraints outside Bedrock’s supported subset.

Build a test matrix for each field:

• Happy path — each valid enum value
• Enum violations — free-text values, wrong case ("OPEN"), empty string, injection attempt
• Type violations — string where number expected (confidence: "high"), bool where float expected
• Missing required fields — each required field absent individually; all absent

For each failure case, verify:

• Response type is schema_violation (not an unhandled exception)
• reason field names the exact field and the violated constraint

For Bedrock Structured Outputs specifically: unit-test the local mock’s schema logic. Trust the real Bedrock service for constrained decoding guarantees — test the Lambda’s error-handling path separately.

Less than people expect, because the enforcement moved from your code to Bedrock’s generation step. The schema compiles into a grammar once and Bedrock caches it for 24 hours — you’re not paying a re-compilation cost per request, just normal Converse call volume.

What genuinely needs attention at 10k rpm:

• Provisioned Throughput or a quota increase — same constraint as any high-volume Bedrock workload, not specific to structured outputs.
• The Pydantic fallback path, if you’re running the hybrid pattern — post-hoc validation for range checks (confidence in [0.0, 1.0]) runs in your own compute, so it scales with your Lambda/ECS concurrency, not with Bedrock.
• Schema-violation logging volume — even a low violation rate (say 0.5%) is 50 structured error events per minute at this scale; make sure the CloudWatch metric filter grouping by violated field doesn’t get lost in noise.

The one-line answer: constrained decoding pushes the expensive part (guaranteeing valid output) into a cached, one-time grammar compile — it’s the retry elimination, not raw throughput, that’s the real cost story at scale. A system doing post-hoc-only validation at 10k rpm would be paying for retries on every rejected generation; this pattern mostly avoids that cost by construction.

“Think of a dropdown menu on a form. You can’t type ‘still under investigation’ in a dropdown — you can only pick from the options the form gives you.

Bedrock Structured Outputs does the same thing for the AI: instead of a free-text field, we give it a dropdown with exactly three options — open, under_review, or resolved. It has to pick one. It can’t write anything else.

We’re switching the status field from a text box to a dropdown. Once that’s deployed, the misrouting stops.”

Technical one-liner: Bedrock Structured Outputs compiles the JSON Schema into a generation grammar — the invalid token can’t be sampled. The routing queue never sees it.