D7 · Production Safety and Operations

This domain is about 8% of the OAI-API mock – roughly 5 of 60 items – and draws on the operations and safety material threaded through the Academy API pathway. It tests whether you can take a working application to production safely: moderating content, red-teaming and monitoring for misalignment, marking provenance, setting rate and spend limits, handling errors with retries, and locking down access with RBAC and network controls.

What you need to know

Production is where an application meets untrusted inputs, real money and real users. You screen content with moderation and safety classifiers, probe weaknesses with red teaming, and watch for misalignment in live behaviour. You mark AI-generated media with content provenance. You cap blast radius with rate limits and spend limits, and you handle transient failures with retries and exponential backoff keyed off the right error codes. You restrict who and what can reach the API with RBAC, Private Link, IP allowlists, mutual TLS and workload identity federation – so credentials are scoped, short-lived and network-bounded. A deployment checklist ties these together as launch gates.

Learning objectives

By the end of this page you should be able to:

1. Apply moderation and safety classifiers to inputs and outputs.
2. Explain red teaming and misalignment monitoring.
3. Set rate limits and spend limits to bound blast radius.
4. Handle error codes with correct retry and backoff behaviour.
5. Choose access controls: RBAC, Private Link, IP allowlist, mTLS, workload identity federation.
6. Run a deployment checklist before launch.

7.1 Safety: moderation, classifiers, red teaming, misalignment

Red teaming is proactive (you attack your own system); misalignment monitoring is continuous (you watch what it actually does). Content provenance (marking AI-generated media) supports transparency obligations.

7.2 Rate limits and spend limits

Two different blast-radius controls:

Runaway agent loops calling the API 1000×/min
        │
   rate limit → 429 caps the request rate
   spend limit → hard cap on dollars, alerting before the ceiling

Set both before launch. A spend limit is the difference between a bug costing $50 and costing $50,000.

7.3 Error codes and retries

import time
for attempt in range(5):
    try:
        return client.responses.create(model="gpt-5.6-terra", input=q)
    except RateLimitError:               # 429
        time.sleep(2 ** attempt)         # 1, 2, 4, 8, 16s
    except BadRequestError:              # 400 — do not retry
        raise

The discriminator on many items: retry transient errors (429/5xx) with backoff; never blindly retry a 400/401/403 – retrying a malformed or unauthorised request just wastes quota and can worsen rate limiting.

7.4 Access control and network security

7.5 The deployment checklist

Before launch, confirm:
[ ] Moderation on untrusted inputs and outputs
[ ] Red-team pass for injection / jailbreak / exfiltration
[ ] Guardrails + human approval on irreversible actions
[ ] Rate limit set; spend limit set with alerting below the ceiling
[ ] Retry with exponential backoff on 429/5xx; no retry on 4xx auth/validation
[ ] RBAC least privilege on keys and roles
[ ] Network controls as required (Private Link / IP allowlist / mTLS)
[ ] Federated, short-lived credentials instead of static keys where possible
[ ] Data residency / retention verified against requirements (see D4)
[ ] Eval gate green (see D3); tracing/observability on (see D4)
[ ] Misalignment monitoring for deployed agents

Decision framework

Use GUARD to decide the controls a production launch needs.

Common mistakes

Scenario challenge

Scenario. You are launching an autonomous support agent that reads customer messages (untrusted input), calls internal tools to issue refunds, and runs on Kubernetes. Security asks: how do we stop it draining budget, stop a malicious message hijacking it, and avoid static API keys in the cluster? The team’s current plan wraps every API call in a “retry up to 5 times on any error” loop and stores a long-lived API key in a Kubernetes secret.

Expert reasoning trace.

1. Budget blast radius → spend limit + rate limit. An agent that loops or is abused could call the API thousands of times. Set a spend limit with alerting below the ceiling and a rate limit, so a runaway loop hits a hard dollar cap and a 429 rather than an unbounded bill.
2. Malicious message → red team + moderation + guardrails + approval. Customer messages are untrusted, so a prompt-injection payload could try to make the agent issue an unauthorised refund. Red-team for injection before launch, moderate inputs, add output/tool guardrails, and gate the refund action behind human approval (D4) since it is irreversible.
3. Retry loop is wrong. “Retry any error 5×” will happily retry a 400 (malformed) and a 401/403 (auth/permission), wasting quota and masking real bugs. Correct: retry only 429/5xx with exponential backoff, and fail fast on 4xx auth/validation.
4. Static key is a liability. A long-lived key in a K8s secret can leak. Use workload identity federation so the pod exchanges its Kubernetes identity for a short-lived token – no static key to steal.
5. Network controls. If the workload’s egress is fixed, add an IP allowlist, and use Private Link if the traffic must stay off the public internet; RBAC scopes the credential to only the refund/support tools.
6. Monitor after launch. Turn on misalignment monitoring and tracing so a drifting or hijacked agent is caught in production.

Exam-correct decision: spend + rate limits, red teaming and moderation with an approval gate on refunds, selective retry (429/5xx with backoff only), workload identity federation instead of a static key, RBAC and network controls, and misalignment monitoring. Not retry-everything, not a long-lived key in a secret, not an auto-refund with no gate.

Assessment traps

Practice questions

Each item states how many responses to select. Commit before revealing.

Key takeaways

• Screen untrusted inputs and model outputs with moderation and safety classifiers; red-team before launch.
• Set both a rate limit (throughput) and a spend limit (dollars) with alerting to bound blast radius.
• Retry only transient errors (429, 5xx) with exponential backoff; never blindly retry 400/401/403.
• Scope credentials with RBAC least privilege and add Private Link, IP allowlist or mTLS to the risk level.
• Prefer workload identity federation and short-lived tokens over long-lived static keys.
• Gate irreversible agent actions behind human approval and monitor deployed agents for misalignment.
• Treat the deployment checklist as launch gates: safety, limits, resilience, access, residency, evals and observability.