# D1 · Applications and Integration
Building on the Messages API in depth – request/response anatomy, streaming, thinking, prompt caching, batching, errors, rate limits, SDKs, third-party access, software-engineering foundations, application design and configuration management.
import { Accordions, AccordionItem, Tabs, TabItem, Steps } from '@prosefly/astro-components';
This is by far the heaviest domain on the Developer exam – roughly **18 of 53 items**. It tests whether you can integrate Claude *correctly*: knowing the exact shape of a Messages API request and response, handling every `stop_reason`, streaming with SSE, using vision/PDF/Files inputs, wiring prompt caching and batching for cost, dealing with errors and rate limits, and structuring an application so that instructions, configuration and secrets live in the right places. Most items are code-shaped scenarios where one option is subtly wrong about an API mechanic.
## Learning objectives
By the end of this page you should be able to:
1. Describe the **anatomy of a Messages API request and response**, including roles, content blocks, `system`, `max_tokens`, `temperature`/`top_p`, `stop_sequences` and `usage`.
2. Handle every **`stop_reason`** value, including `tool_use`, `pause_turn`, `refusal` and `max_tokens`.
3. Maintain **multi-turn history** correctly and stream responses using the **SSE event types** in Python and TypeScript.
4. Send **vision, PDF and Files API** inputs, and enable **extended / adaptive thinking**.
5. Apply **prompt caching** with `cache_control` and compute the cost impact.
6. Use the **Message Batches API** lifecycle and its 50% discount.
7. Handle **error codes**, implement **retry with backoff + jitter**, use **idempotency**, and reason about **rate limits** (RPM/ITPM/OTPM) and **timeouts**.
8. Access Claude through **Bedrock, Vertex AI and Foundry**, and use the **Python and TypeScript SDKs** including async patterns.
9. Apply **software-engineering foundations** (REST, JSON, async, version control, refactoring) and sound **application design** and **configuration management**.
---
## 1.1 Requirements and the application lifecycle
Before any code, an integration has a lifecycle: **define requirements → prototype → evaluate → harden → deploy → monitor → iterate.** The exam expects you to know where Claude fits and what changes at each stage.
| Stage | Key decisions | Claude-specific concerns |
| --- | --- | --- |
| Requirements | Task, quality bar, latency budget, cost ceiling, data sensitivity | Which model tier; sync vs batch; ZDR needs |
| Prototype | Happy-path prompt, model, output shape | Pin a snapshot; capture example inputs/outputs |
| Evaluate | Golden set, metrics, per-segment accuracy | LLM-as-judge in a *separate* session; temperature 0 for reproducibility |
| Harden | Errors, retries, timeouts, rate limits, validation | Backoff + jitter; schema validation-retry; hooks for critical rules |
| Deploy | Secrets, config, observability | Keys in secret manager; log request IDs; pin model version |
| Monitor / iterate | Drift, cost, latency, failures | Track `usage`, cache hit rate, `stop_reason` distribution |
:::tip[Exam signal]
Words like "before production", "reliability", "reproducible", "cost ceiling" or "SLA" push you toward hardening concerns – retries, timeouts, pinning, validation – not toward prompt wording.
:::
---
## 1.2 Anatomy of a Messages API request
A Messages request is a JSON body sent to `POST /v1/messages`. The core fields:
```json
{
"model": "claude-sonnet-5",
"max_tokens": 1024,
"system": "You are a precise assistant. Answer only from the provided context.",
"messages": [
{ "role": "user", "content": "Summarise the attached report in 3 bullets." }
],
"temperature": 0.2,
"stop_sequences": ["\n\nHuman:"]
}
```
Key fields:
- **`model`** – a model ID; pin a snapshot in production (see 1.16).
- **`max_tokens`** – the maximum tokens Claude may *generate* (not the context window). Required. If output hits it, `stop_reason` is `max_tokens`.
- **`system`** – a top-level string (or array of blocks) for role/instructions. It is **not** a message with `role: "system"` in the `messages` array on current models (Sonnet 5 has no mid-conversation system messages).
- **`messages`** – an alternating list of `user` and `assistant` turns. Each has `role` and `content`.
- **`temperature`** (0–1) and **`top_p`** – sampling controls. Set **one**, not both. Lower temperature = more deterministic; `temperature: 0` for maximum reproducibility.
- **`stop_sequences`** – strings that, if generated, halt output; `stop_reason` becomes `stop_sequence`.
### Roles and content blocks
`content` is either a string (shorthand for a single text block) or an **array of content blocks**. Block types include `text`, `image`, `document`, `tool_use`, `tool_result`, and `thinking`.
```json
{
"role": "user",
"content": [
{ "type": "text", "text": "What is in this image?" },
{ "type": "image", "source": { "type": "base64", "media_type": "image/png", "data": "iVBORw0KG..." } }
]
}
```
:::note[Roles are strict]
`messages` must start with `user` and alternate. The assistant's prior replies (including `tool_use` blocks) go back verbatim as `role: "assistant"`; tool outputs go back as `role: "user"` with `tool_result` blocks. Getting the roles wrong is a common `400 invalid_request`.
:::
---
## 1.3 Anatomy of a Messages API response
```json
{
"id": "msg_01ABC...",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-5",
"content": [
{ "type": "text", "text": "Here are three bullets: ..." }
],
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 2145,
"output_tokens": 87,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0
}
}
```
- **`id`** – log this (and the `request-id` response header) for support and debugging.
- **`content`** – array of output blocks; iterate rather than assuming a single text block (a response may contain `thinking`, `text` and `tool_use` blocks together).
- **`stop_reason`** – why generation stopped (see 1.4).
- **`usage`** – token accounting, including cache fields. Bill and budget from this.
:::tip[Exam signal]
If an option assumes `response.content[0].text` always exists, be suspicious. With thinking or tools enabled, `content[0]` may be a `thinking` or `tool_use` block. Correct code iterates and filters by `type`.
:::
---
## 1.4 `stop_reason` – the control signal
`stop_reason` is the single most important field for control flow. Never infer termination from the text.
| `stop_reason` | Meaning | Correct handling |
| --- | --- | --- |
| `end_turn` | Claude finished naturally | Return the answer |
| `tool_use` | Claude wants a tool run | Execute tool(s), append `tool_result`, call again |
| `max_tokens` | Hit `max_tokens` cap | Output is truncated; raise cap or continue, do not treat as complete |
| `stop_sequence` | Hit a `stop_sequences` string | Check `stop_sequence` field for which one |
| `pause_turn` | Long-running turn paused (e.g., server tools) | Send the response back unchanged to resume |
| `refusal` | Claude declined for safety | Do not retry blindly; surface/handle per policy |
```python
resp = client.messages.create(model="claude-sonnet-5", max_tokens=1024, messages=msgs)
if resp.stop_reason == "tool_use":
handle_tools(resp) # execute, append tool_result, loop
elif resp.stop_reason == "pause_turn":
msgs.append({"role": "assistant", "content": resp.content})
resp = client.messages.create(model="claude-sonnet-5", max_tokens=1024, messages=msgs)
elif resp.stop_reason == "max_tokens":
handle_truncation(resp) # output is incomplete
elif resp.stop_reason == "refusal":
handle_refusal(resp) # policy path, not a retry loop
```
:::danger[Anti-pattern #1]
Parsing the assistant's prose ("It looks like I'm done", "I'll now stop") to decide whether to stop is anti-pattern #1. Drive the loop from `stop_reason`.
:::
---
## 1.5 Multi-turn conversations
State is client-side: you resend the whole history each turn. Append the assistant's response verbatim, then the next user turn.
```python
messages = [{"role": "user", "content": "My name is Dana."}]
r1 = client.messages.create(model="claude-sonnet-5", max_tokens=256, messages=messages)
messages.append({"role": "assistant", "content": r1.content}) # append full blocks
messages.append({"role": "user", "content": "What is my name?"})
r2 = client.messages.create(model="claude-sonnet-5", max_tokens=256, messages=messages)
```
Because history grows every turn, so does input cost and latency. This is why **prompt caching** (1.9), **context editing** and **compaction** matter for long conversations.
:::caution[Fable 5.1 is append-only]
On `claude-fable-5-1`, editing, reordering or removing earlier turns invalidates later `thinking` blocks. Harnesses must be **append-only**: freeze `system` and `tools`, put mid-session changes in `role: "system"` messages where supported, and trim server-side via context editing / compaction rather than mutating history.
:::
---
## 1.6 Streaming with SSE
Streaming returns Server-Sent Events so you can render tokens as they arrive. The event sequence:
```text
message_start
content_block_start (index 0)
content_block_delta ... (text_delta / input_json_delta / thinking_delta)
content_block_stop
[more content blocks ...]
message_delta (carries stop_reason and final usage)
message_stop
```
```python
from anthropic import Anthropic
client = Anthropic()
with client.messages.stream(
model="claude-sonnet-5",
max_tokens=1024,
messages=[{"role": "user", "content": "Write a haiku about tokens."}],
) as stream:
for text in stream.text_stream: # convenience: text deltas only
print(text, end="", flush=True)
final = stream.get_final_message() # full Message with stop_reason + usage
print("\n", final.stop_reason, final.usage.output_tokens)
```
```typescript
import Anthropic from '@anthropic-ai/sdk';
const client = new Anthropic();
const stream = client.messages.stream({
model: 'claude-sonnet-5',
max_tokens: 1024,
messages: [{ role: 'user', content: 'Write a haiku about tokens.' }],
});
stream.on('text', (delta) => process.stdout.write(delta));
const final = await stream.finalMessage();
console.log('\n', final.stop_reason, final.usage.output_tokens);
```
```python
with client.messages.stream(model="claude-sonnet-5", max_tokens=512,
messages=[{"role": "user", "content": "hi"}]) as stream:
for event in stream:
if event.type == "content_block_delta":
if event.delta.type == "text_delta":
print(event.delta.text, end="")
elif event.delta.type == "input_json_delta":
print(event.delta.partial_json, end="") # tool args stream as JSON
elif event.type == "message_delta":
print("\nstop:", event.delta.stop_reason)
```
:::tip[Exam signal]
"Show output as it is generated", "improve perceived latency", "long response" → streaming. Remember tool arguments arrive as `input_json_delta` (partial JSON) and final `stop_reason`/`usage` arrive on `message_delta`.
:::
---
## 1.7 Vision, PDF and the Files API
Multimodal inputs are content blocks in a `user` message.
```json
{
"role": "user",
"content": [
{ "type": "text", "text": "Extract the invoice total." },
{ "type": "image", "source": { "type": "url", "url": "https://example.com/invoice.png" } },
{ "type": "document", "source": { "type": "base64", "media_type": "application/pdf", "data": "JVBERi0..." } }
]
}
```
- **Images**: `source.type` may be `base64` or `url`. Supported types include PNG, JPEG, GIF, WebP.
- **PDFs**: `type: "document"` with `application/pdf`; Claude reads text and page images.
- **Files API**: upload large or reused files once, then reference by `file_id` instead of re-sending bytes each turn – saves upload bandwidth and enables reuse.
```python
uploaded = client.files.upload(file=("report.pdf", open("report.pdf", "rb"), "application/pdf"))
resp = client.messages.create(
model="claude-sonnet-5", max_tokens=1024,
messages=[{"role": "user", "content": [
{"type": "text", "text": "Summarise."},
{"type": "document", "source": {"type": "file", "file_id": uploaded.id}},
]}],
)
```
**Citations** can be enabled on documents so Claude returns grounded references to source spans.
---
## 1.8 Extended and adaptive thinking
Thinking lets Claude reason before answering; the reasoning appears as `thinking` content blocks.
```json
{
"model": "claude-opus-5",
"max_tokens": 4096,
"thinking": { "type": "adaptive" },
"messages": [{ "role": "user", "content": "Prove sqrt(2) is irrational." }]
}
```
- **All current models** accept `thinking: {"type": "adaptive"}`.
- **`budget_tokens`** is only valid on **Haiku 4.5**; it returns `400` on Fable 5.x / Opus 5 / Sonnet 5.
- **Effort levels** `low | medium | high (default) | xhigh` tune reasoning depth (`xhigh` for the hardest coding/agentic work on Opus 5 / Fable 5.1). Haiku 4.5 has no `effort` parameter.
- **Fable 5.1** always has thinking on; its thinking blocks are readable only by the producing model or newer (a silent fallback to an older model drops them).
```python
# Haiku 4.5 – the only current model using budget_tokens
client.messages.create(
model="claude-haiku-4-5", max_tokens=2048,
thinking={"type": "enabled", "budget_tokens": 1024},
messages=[{"role": "user", "content": "Plan the refactor."}],
)
```
:::caution[Preserve thinking blocks]
When continuing a conversation that used thinking, append the assistant's `thinking` blocks back verbatim. Stripping them can break tool-use continuations and, on Fable 5.1, invalidate later turns.
:::
---
## 1.9 Prompt caching mechanics and cost math
Prompt caching stores a prefix of the request so repeated calls skip re-processing it. Mark the **end** of the stable prefix with `cache_control`.
```json
{
"model": "claude-sonnet-5",
"max_tokens": 512,
"system": [
{ "type": "text", "text": "You are a support agent. Policies:\n...large...",
"cache_control": { "type": "ephemeral" } }
],
"messages": [{ "role": "user", "content": "How do I return an item?" }]
}
```
Rules:
- Put **stable content first** (system prompt, tool definitions, long documents), then variable content.
- Minimum cacheable prefix is **~1024 tokens** (2048 on Haiku).
- **Cache write** costs ≈ **1.25×** base input (5-minute TTL) or **2×** (1-hour TTL).
- **Cache read** costs ≈ **0.1×** base input (10% of the price).
- Reported in `usage` as `cache_creation_input_tokens` and `cache_read_input_tokens`.
### Worked cost example
A support bot sends a 10,000-token cached policy prefix on Sonnet 5 (input $2/MTok) plus 200 variable tokens, 100 calls/hour.
| Scenario | Prefix cost per call | Notes |
| --- | --- | --- |
| No caching | 10,000 × $2 / 1e6 = **$0.0200** | Reprocessed every call |
| First call (write, 5-min) | 10,000 × $2 × 1.25 / 1e6 = **$0.0250** | Pay once |
| Cache hits (reads) | 10,000 × $2 × 0.1 / 1e6 = **$0.0020** | 90% cheaper on the prefix |
Over 100 calls: no-cache ≈ **$2.00** on the prefix; cached ≈ $0.025 + 99 × $0.002 ≈ **$0.223** – roughly a **9×** reduction on the cached portion.
:::tip[Exam signal]
"Same large instructions/documents on every call", "reduce input cost", "high request volume with a shared prefix" → prompt caching. If the prefix changes every call, caching does not help.
:::
---
## 1.10 Message Batches API
For latency-tolerant, high-volume work, the Batches API processes many requests asynchronously at a **50% discount** on input and output tokens, with results typically well within 24 hours.
1. **Create** a batch with a list of requests, each with a `custom_id`.
```python
batch = client.messages.batches.create(requests=[
{"custom_id": "row-1", "params": {"model": "claude-haiku-4-5", "max_tokens": 256,
"messages": [{"role": "user", "content": "Classify: great product"}]}},
{"custom_id": "row-2", "params": {"model": "claude-haiku-4-5", "max_tokens": 256,
"messages": [{"role": "user", "content": "Classify: terrible support"}]}},
])
```
2. **Poll** `processing_status` until it is `ended`.
```python
import time
while client.messages.batches.retrieve(batch.id).processing_status != "ended":
time.sleep(30)
```
3. **Stream results** and match by `custom_id`.
```python
for result in client.messages.batches.results(batch.id):
print(result.custom_id, result.result.type) # "succeeded" | "errored" | "expired"
```
:::tip[Exam signal]
"Overnight", "nightly classification of thousands of records", "not latency-sensitive", "cut cost in half" → Message Batches. If a user is waiting in real time, batch is wrong.
:::
---
## 1.11 Error codes, retries and idempotency
| Status | Type | Retry? |
| --- | --- | --- |
| 400 | `invalid_request_error` | No – fix the request |
| 401 | `authentication_error` | No – fix the key |
| 403 | `permission_error` | No |
| 404 | `not_found_error` | No |
| 413 | `request_too_large` | No – shrink the request |
| 429 | `rate_limit_error` | Yes – backoff, respect `retry-after` |
| 500 | `api_error` | Yes – backoff |
| 529 | `overloaded_error` | Yes – backoff |
Retry `429`, `500` and `529` with **exponential backoff + jitter**; do not retry `4xx` other than `429`.
```python
import time, random
from anthropic import Anthropic, APIStatusError, RateLimitError
client = Anthropic(max_retries=0) # disable SDK auto-retry to show the pattern
def call_with_backoff(**kwargs):
for attempt in range(6):
try:
return client.messages.create(**kwargs)
except (RateLimitError, APIStatusError) as e:
status = getattr(e, "status_code", None)
if status not in (429, 500, 529):
raise
retry_after = float(getattr(e, "response", None).headers.get("retry-after", 0)) if getattr(e, "response", None) else 0
sleep = max(retry_after, min(60, (2 ** attempt))) + random.uniform(0, 1) # jitter
time.sleep(sleep)
raise RuntimeError("exhausted retries")
```
The SDKs retry safely by default (`max_retries=2`). For **idempotency** on writes (e.g., batch creation), pass an idempotency key so a retried request is not processed twice.
:::note[Log the request ID]
Every response carries a `request-id`. Log it with your own correlation ID; Anthropic support and your traces both key off it. This directly supports Domain 8 debugging.
:::
---
## 1.12 Rate limits and timeouts
Rate limits are enforced per model and tier along three axes:
| Limit | Meaning |
| --- | --- |
| **RPM** | Requests per minute |
| **ITPM** | Input tokens per minute |
| **OTPM** | Output tokens per minute |
You may hit any one first. `429` responses carry `retry-after` and rate-limit headers. Strategies: client-side rate limiting/queueing, spreading load, batching, requesting a higher tier, and reducing tokens (shorter output, caching). Use `client.models.list()` / `.retrieve(id)` for live limits.
Set **timeouts** deliberately – long thinking or large outputs need generous timeouts; short interactive calls should fail fast. The SDKs expose a `timeout` option.
```typescript
const client = new Anthropic({ timeout: 60_000, maxRetries: 3 });
```
---
## 1.13 Third-party access: Bedrock, Vertex, Foundry
Claude is available through three cloud platforms in addition to the Anthropic API. The Messages API shape is the same; auth, model IDs and region differ.
| Platform | SDK | Auth | Notes |
| --- | --- | --- | --- |
| Anthropic API | `anthropic` / `@anthropic-ai/sdk` | `ANTHROPIC_API_KEY` | Full, earliest feature access |
| Amazon Bedrock | `AnthropicBedrock` | AWS IAM / SigV4 | FedRAMP High available; Bedrock model IDs |
| Google Vertex AI | `AnthropicVertex` | GCP ADC / service account | Vertex model IDs, region-scoped |
| Microsoft Foundry | Foundry SDK / API | Entra ID | Azure-native governance |
```python
from anthropic import AnthropicBedrock
client = AnthropicBedrock(aws_region="us-east-1")
resp = client.messages.create(model="anthropic.claude-sonnet-5",
max_tokens=512,
messages=[{"role": "user", "content": "Hello"}])
```
:::tip[Exam signal]
"Data must stay in our AWS/GCP/Azure account", "FedRAMP", "existing cloud governance" → Bedrock / Vertex / Foundry. The code differs mainly in the client constructor and model IDs.
:::
---
## 1.14 SDKs and async patterns
```python
from anthropic import Anthropic
client = Anthropic() # reads ANTHROPIC_API_KEY from env
resp = client.messages.create(model="claude-sonnet-5", max_tokens=256,
messages=[{"role": "user", "content": "Hi"}])
print(resp.content[0].text)
```
```python
import asyncio
from anthropic import AsyncAnthropic
client = AsyncAnthropic()
async def classify(text: str) -> str:
r = await client.messages.create(model="claude-haiku-4-5", max_tokens=64,
messages=[{"role": "user", "content": f"Label: {text}"}])
return r.content[0].text
async def main():
results = await asyncio.gather(*[classify(t) for t in ["a", "b", "c"]])
print(results)
asyncio.run(main())
```
```typescript
import Anthropic from '@anthropic-ai/sdk';
const client = new Anthropic();
const results = await Promise.all(
['a', 'b', 'c'].map((t) =>
client.messages.create({
model: 'claude-haiku-4-5',
max_tokens: 64,
messages: [{ role: 'user', content: `Label: ${t}` }],
}),
),
);
console.log(results.map((r) => r.content[0].type));
```
Use **async / concurrency** to parallelise independent calls (respecting rate limits), never to fake ordering between dependent calls. For thousands of independent items that can wait, prefer **batching** over hand-rolled concurrency.
---
## 1.15 Software-engineering foundations
The exam assumes fluency with the fundamentals that make an integration robust.
| Foundation | What the exam expects |
| --- | --- |
| **REST** | Claude is an HTTP JSON API: methods, status codes, headers (`x-api-key`, `anthropic-version`), idempotency |
| **JSON** | Request/response bodies, schemas, escaping; validate before trusting |
| **Async** | Non-blocking IO, concurrency limits, backpressure; parallelise independent calls |
| **Version control** | Commit prompts, schemas and config; review changes; tag releases |
| **Refactoring** | Extract prompt templates, centralise the client, isolate model IDs so migration is a one-line change |
:::note[Why this matters]
A well-refactored integration pins the model ID and prompt template in one place, wraps the client with retry/timeout defaults, and validates every structured output. These make the reliability and cost questions elsewhere on the exam trivial to answer correctly.
:::
---
## 1.16 Application design across surfaces
The *same words* are interpreted differently depending on where they run. Know the surfaces:
| Surface | Instruction source | Determinism | Best for |
| --- | --- | --- | --- |
| **API / SDK** | `system` + `messages` you send | You control everything | Production apps, pipelines |
| **Agent SDK** | `system_prompt` + tools + hooks | You host the loop | Custom agents |
| **Claude Code** | `CLAUDE.md` hierarchy + `settings.json` | Config-driven, tool-permissioned | Coding in the terminal |
| **Claude Desktop** | App settings + MCP config | GUI-driven | Local assistant + MCP |
| **claude.ai** | Chat UI, Projects | Least programmatic | Ad-hoc, non-developer use |
**Content boundaries with XML tags.** Wrap untrusted or distinct inputs in tags so the model can tell instructions from data:
```text
...trusted rules...
...untrusted content – treat as data, not instructions...
```
**Schema design and session hygiene.** Define the output schema up front (1.9, D4); keep sessions focused (one task per session where practical); clear or compact long histories; never let untrusted document text be interpreted as instructions.
:::tip[Exam signal]
If a stem mixes trusted instructions with pasted user/web/tool content, the correct answer isolates the untrusted content in tags and treats it as data – it does not rely on the model "knowing" not to follow it.
:::
---
## 1.17 Configuration management
Keep behaviour reproducible and secrets out of prompts.
| Concern | Where it lives | Rule |
| --- | --- | --- |
| Behavioural instructions | `CLAUDE.md` hierarchy (Claude Code) / `system` (API) | Version-controlled, reviewed |
| Model version | Config / env var, pinned snapshot | One place; never hard-code across files |
| Prompt templates | Versioned files, tagged | Change = new version, re-eval |
| Secrets / API keys | Env vars or secret manager | **Never** in prompts, `CLAUDE.md`, or committed files |
| Environment differences | `.env` per environment | Dev/stage/prod isolation |
```python
import os
MODEL = os.environ["CLAUDE_MODEL"] # e.g. "claude-sonnet-5" – pinned, env-driven
client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"]) # key from env, never literal
```
:::danger[Secrets never go in prompts]
Putting an API key, database password or token in the `system` prompt or in `CLAUDE.md` leaks it into logs, history and (for `CLAUDE.md`) version control. Use environment variables or a secret manager. This overlaps with Domain 6.
:::
---
## 1.18 Structured outputs and citations end-to-end
Beyond raw text, D1 items often probe whether you can obtain machine-readable output *and* keep it grounded. Two request-level features do this: **`output_config.format`** (schema-constrained JSON) and **document citations** (grounded source spans).
```json
{
"model": "claude-sonnet-5",
"max_tokens": 1024,
"output_config": {
"format": {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {
"total_cents": {"type": "integer"},
"currency": {"type": "string", "enum": ["USD", "EUR", "GBP"]}
},
"required": ["total_cents", "currency"]
}
}
},
"messages": [{"role": "user", "content": [
{"type": "document", "source": {"type": "file", "file_id": "file_01ABC"},
"citations": {"enabled": true}},
{"type": "text", "text": "Extract the invoice total."}
]}]
}
```
A grounded response can carry `citations` on its text blocks referencing the source spans:
```json
{
"content": [
{"type": "text", "text": "The total is $482.10.",
"citations": [{"type": "page_location", "cited_text": "Total due: $482.10",
"document_index": 0, "start_page_number": 3, "end_page_number": 3}]}
],
"stop_reason": "end_turn"
}
```
| Feature | What it guarantees | What it does *not* do |
| --- | --- | --- |
| `output_config.format` (JSON schema) | Output conforms to the schema shape | Guarantee the *values* are correct — still validate semantics |
| `strict: true` tool schema | Tool input matches the schema exactly | Work with forced `tool_choice` on Fable 5.1 (that 400s) |
| Document `citations` | Text blocks reference the source spans they used | Prevent hallucination if the source itself is wrong |
:::tip[Exam signal]
'Must return schema-valid JSON' → `output_config.format` (schema) or a `strict: true` tool, **plus validation-retry**. 'Must show where each claim came from' → enable document `citations`. Schema conformance is not the same as value correctness — the exam rewards validating both.
:::
---
## 1.19 Idempotency, timeouts and the SDK retry contract
Writes and long calls need explicit reliability controls. The SDKs retry transient errors by default, but you own idempotency and timeout budgets.
| Control | Why it matters | How |
| --- | --- | --- |
| **Idempotency key** | A retried create (e.g. a batch) must not run twice | Pass an idempotency key on write requests |
| **Timeout** | Long thinking / large output needs headroom; interactive calls should fail fast | Set `timeout` per call class |
| **Bounded retries** | Recover from `429`/`5xx`/`529` without amplifying load | SDK `max_retries` (default 2) + jitter |
| **Concurrency cap** | Prevents self-inflicted rate-limit storms | Semaphore / queue around the client |
```python
from anthropic import Anthropic
client = Anthropic(max_retries=3, timeout=120) # bounded retries; generous timeout for batch
batch = client.messages.batches.create(
requests=[...],
extra_headers={"Idempotency-Key": "nightly-2026-09-15"}, # safe to retry, runs once
)
```
```typescript
import Anthropic from '@anthropic-ai/sdk';
const client = new Anthropic({ maxRetries: 3 });
const resp = await client.messages.create(
{ model: 'claude-sonnet-5', max_tokens: 8000, messages },
{ timeout: 120_000 }, // long output → longer timeout; short calls stay fast
);
```
:::tip[Exam signal]
'A retried write ran twice' → **idempotency key**. 'Long-output/thinking call times out but the model was fine' → raise the **timeout**, do not just retry. 'Retries make the 429 worse' → cap concurrency and honour `retry-after`.
:::
---
## 1.20 Common misconceptions
| Misconception | Reality | Why it matters on the exam |
| --- | --- | --- |
| The API remembers the conversation server-side | The Messages API is stateless; you resend history each turn | Explains why history/caching/token cost grow, and why 'send only the latest message' is wrong |
| `max_tokens` is the context window | It caps *generated output* only, within the window | Distinguishes `max_tokens` truncation from a 413 request-too-large |
| `response.content[0].text` always holds the answer | With thinking/tools, block 0 may be `thinking`/`tool_use`; iterate by type | The most common code-shaped distractor in D1 |
| Any error should be retried with backoff | Only `429`/`500`/`529` are transient; `4xx` (except 429) are deterministic | Retrying a `400`/`401` loops forever and hides the real fix |
| Streaming makes generation faster | It only improves time-to-first-token; total time is unchanged | Separates a perceived-latency fix from a real-latency fix |
| Prompt caching helps any repeated request | Only a byte-identical prefix above the minimum caches | A per-request timestamp or user name in the prefix kills the cache |
| Bedrock/Vertex require rewriting the request | Only the client/auth and model ID change; the body is portable | Migration questions hinge on knowing what actually changes |
| A refusal is an error to retry | `refusal` is a deliberate safety stop reason; route to policy | Prevents blind-retry loops on safety declines |
---
## 1.21 Scenario walkthrough: a resilient extraction service
**Scenario.** You own a service that extracts structured fields from up to 100,000 uploaded PDFs per night. Each request sends a 6,000-token instruction+schema prefix (identical every call) plus one document; results are needed by 08:00, not in real time. During the day, a low-volume interactive endpoint answers ad-hoc questions about a single reused 180-page contract. Recently the nightly job started failing intermittently with `429`s and occasional `JSONDecodeError`s, and finance flagged the cost as too high. You must make it reliable and cheap without hurting extraction quality (Sonnet 5 currently clears the bar).
**Expert reasoning trace.**
1. **Classify each workload by latency tolerance.** The nightly job is latency-tolerant and bulk → it belongs on the **Message Batches API** (50% off, results within 24h), not hand-rolled concurrency that triggers `429` storms. The interactive endpoint is real-time → keep it synchronous.
2. **Attack cost with the right levers, in order.** Cheapest model that clears the bar is already chosen (Sonnet 5 — do *not* jump to Opus 5, which is over-engineered here). Then **cache the 6,000-token stable prefix** (drops it to ~10% on hits) and run through **Batches** (another 50%). Stacking model + cache + batch is the intended answer; picking only one leaves savings on the table.
3. **Fix the `429`s at the source, not with tighter retries.** Moving to Batches removes most of the pressure; where synchronous calls remain, cap concurrency and honour `retry-after`. Retrying immediately (a tempting distractor) worsens the limit.
4. **Fix the `JSONDecodeError` in the correct layer.** This is a *model-output/parsing* problem, not transport. Use **`output_config.format` with a JSON schema plus validation-retry** that feeds the error back — not backoff, which is for transient transport errors.
5. **Handle the reused contract efficiently.** Upload it once via the **Files API** and reference by `file_id`; re-sending 180 pages of base64 each call is the bandwidth/cost trap.
6. **Reject the tempting alternatives.** 'Move everything to Opus 5 for quality' — constraint-blind and costly. 'Rotate API keys to beat the 429' — limits are per account, not per key. 'Increase `max_tokens` to fix the JSON errors' — wrong layer; that addresses truncation, not malformed JSON.
**Correct decision.** Batches API on Sonnet 5 with `cache_control` on the shared prefix and idempotency keys on batch creation; schema-constrained output with validation-retry for the JSON errors; Files API for the reused contract; concurrency caps and `retry-after` for any remaining synchronous calls.
---
## Exam traps in this domain
| Trap | Why it is wrong |
| --- | --- |
| Reading `stop_reason` from the response text | `stop_reason` is a structured field; text is not a control signal (anti-pattern #1) |
| Assuming `response.content[0].text` always exists | With thinking/tools, `content[0]` may be a `thinking` or `tool_use` block |
| Setting both `temperature` and `top_p` | Set one sampling control, not both |
| Treating `max_tokens` as the context window | `max_tokens` caps *generated* output only |
| Using `budget_tokens` on Sonnet 5 / Opus 5 / Fable 5.1 | Returns `400`; only Haiku 4.5 still uses `budget_tokens` |
| Caching a prefix that changes every call | No cache hits; caching only helps stable prefixes |
| Retrying a `400`/`401` with backoff | Only `429`/`5xx`/`529` are retryable |
| Ignoring `retry-after` on `429` | You will keep hitting the limit; honour the header |
| Putting an API key in the `system` prompt or `CLAUDE.md` | Leaks the secret into logs/history/VCS |
| Mutating earlier turns on Fable 5.1 | Invalidates later thinking blocks; harness must be append-only |
| Using synchronous calls for an overnight bulk job | Batches API gives 50% off for latency-tolerant work |
| Sending PDF bytes on every turn instead of the Files API | Wastes bandwidth; upload once and reference by `file_id` |
| Treating schema-conforming JSON as automatically correct | `output_config.format` guarantees shape, not values; validate semantics too |
| Retrying a `refusal` with backoff | It is a deliberate safety stop reason; route to policy, do not loop |
| Rotating API keys to beat a `429` | Rate limits are per account, not per key; cap concurrency and honour `retry-after` |
| Raising `max_tokens` to fix a `JSONDecodeError` | Wrong layer; malformed JSON is a parsing/model-output problem — use schema + validation-retry |
| Forgetting an idempotency key on a retried batch create | The create can run twice; pass an idempotency key on writes |
---
## Practice questions
Each item states how many responses to select. Attempt before revealing.
A. Add a keyword list ('done', 'finished', 'complete') to catch more cases.
B. Cap the loop at 10 iterations and return whatever is present.
C. Drive the loop from `stop_reason`: continue while it is `tool_use`, stop on `end_turn`.
D. Lower `temperature` so the wording is consistent.
**Answer: C.** Termination must come from the structured `stop_reason` field (anti-pattern #1). Keyword matching (A) is brittle; iteration caps (B) are anti-pattern #2 and hide incomplete work; temperature (D) does not create a reliable signal.
A. Thinking is disabled by default, so enable it.
B. `content[0]` is a `thinking` block; iterate `content` and select blocks where `type == 'text'`.
C. Set `max_tokens` higher.
D. Use streaming instead.
**Answer: B.** With thinking or tools, the content array can begin with a `thinking` or `tool_use` block. Robust code iterates and filters by type. The others do not address the shape of the response.
A. Place the policy first and mark the end with `cache_control: {type: 'ephemeral'}`.
B. Switch `temperature` to 0.
C. Move the policy after the user question.
D. Reuse the cached prefix across requests within the TTL.
E. Increase `max_tokens`.
**Answer: A and D.** Caching a large stable prefix and reusing it on subsequent calls cuts the prefix cost to ~10%. The prefix must come first (C is wrong). Temperature (B) and `max_tokens` (E) do not affect input caching.
A. Fire 50,000 synchronous requests with high concurrency on Opus 5.
B. Use the Message Batches API on Haiku 4.5 for the 50% discount.
C. Use streaming to speed each request.
D. Increase the rate limit tier and loop synchronously.
**Answer: B.** Latency-tolerant bulk work is the textbook Batches case: 50% off, results well within 24h, and Haiku 4.5 is the cheapest tier for simple classification. Streaming (C) does not cut cost; brute-force sync (A, D) is expensive and rate-limited.
A. Retry immediately in a tight loop until it succeeds.
B. Treat 429 as fatal and drop the request.
C. Retry with exponential backoff and jitter, honouring the `retry-after` header.
D. Switch to a different API key.
**Answer: C.** 429 is retryable but only with backoff + jitter and respecting `retry-after`. Tight retry (A) worsens the limit; dropping (B) loses work; rotating keys (D) does not raise the account limit and may violate terms.
A. 400 invalid_request
B. 429 rate_limit
C. 401 authentication
D. 529 overloaded
E. 404 not_found
**Answer: B and D.** Rate-limit and overloaded (and 500) are transient and retryable with backoff. 400/401/404 are client errors that retrying will not fix.
A. `budget_tokens` must be under 1024.
B. Sonnet 5 does not accept `budget_tokens`; it is only valid on Haiku 4.5. Use `thinking: {type: 'adaptive'}`.
C. Thinking is not supported on Sonnet 5.
D. `max_tokens` must exceed `budget_tokens`.
**Answer: B.** `budget_tokens` was removed on current non-Haiku models; only Haiku 4.5 still uses it. Current models use `thinking: {type: 'adaptive'}` (optionally with effort levels). Thinking is supported on Sonnet 5 (C wrong).
A. The model finished; return the text.
B. The output was truncated at the `max_tokens` cap; treat as incomplete and raise the cap or continue the turn.
C. The prompt was too long; shrink the input.
D. Claude refused; go to the refusal path.
**Answer: B.** `max_tokens` means generation was cut off at the output cap; the answer is incomplete. `end_turn` (A) would mean finished; input size (C) triggers 413; refusal (D) is a different `stop_reason`.
A. Anthropic API with an API key stored in AWS Secrets Manager.
B. Amazon Bedrock with the `AnthropicBedrock` client and IAM auth.
C. Google Vertex AI.
D. claude.ai with SSO.
**Answer: B.** Bedrock keeps inference in the customer's AWS account under IAM/SigV4 and offers FedRAMP High. Storing an Anthropic key in Secrets Manager (A) still calls the external Anthropic API. Vertex (C) is GCP; claude.ai (D) is not a programmatic in-account path.
A. Paste the PDF text into every prompt.
B. Send the base64 PDF bytes on every request.
C. Upload once via the Files API and reference it by `file_id` in each request.
D. Convert every page to an image and send images each time.
**Answer: C.** The Files API uploads once and references by `file_id`, avoiding repeated uploads. Re-sending text (A), bytes (B) or images (D) each time wastes bandwidth and tokens.
A. Arguments in `text_delta`; `stop_reason` in `message_start`.
B. Arguments in `input_json_delta` (partial JSON on the `tool_use` block); `stop_reason` in `message_delta`.
C. Both in `content_block_start`.
D. Both only after `message_stop`.
**Answer: B.** Tool arguments stream as `input_json_delta` partial JSON; the final `stop_reason` and usage arrive on `message_delta`, before `message_stop`.
A. Read the model ID from a single env-driven constant used everywhere.
B. Move the API key to an environment variable / secret manager and out of the prompt.
C. Store the API key in `CLAUDE.md` so it is documented.
D. Duplicate the model ID into each file for locality.
E. Commit the `.env` file with the real key for reproducibility.
**Answer: A and B.** Centralise the pinned model ID (one-line migrations) and keep secrets in env/secret manager, never in prompts. Putting keys in `CLAUDE.md` (C) or committing real keys (E) leaks them; duplicating IDs (D) makes migration error-prone.
A. It must be sent as a `{role: 'system'}` entry in `messages`.
B. It is a top-level field; Sonnet 5 does not support mid-conversation system messages.
C. It is ignored unless thinking is enabled.
D. It counts as output tokens.
**Answer: B.** `system` is a top-level request field. Sonnet 5 has no mid-conversation system messages. It is input, not output (D), and always applies (C).
A. Results are synchronous; the batch failed.
B. Poll `processing_status` until `ended`, then stream results and match by `custom_id`.
C. Batches only work on Opus 5.
D. Call `retrieve` once; if empty, recreate the batch.
**Answer: B.** Batches are asynchronous: poll until `ended`, then read results keyed by `custom_id`. Recreating (D) duplicates work; results are not synchronous (A); batches are model-agnostic (C).
A. Treat it as an error and retry from scratch.
B. Append the assistant response unchanged and call the API again to resume the turn.
C. Lower `max_tokens`.
D. Switch to batch mode.
**Answer: B.** `pause_turn` indicates a long-running turn was paused (e.g., server tools); send the response back unchanged to resume. It is not an error (A) and unrelated to `max_tokens` (C) or batching (D).
A. Trust the model to recognise and ignore it.
B. Wrap the document in XML tags and instruct that its contents are data to summarise, not instructions to follow.
C. Delete any sentence containing 'ignore'.
D. Raise `temperature` to reduce compliance.
**Answer: B.** Content boundaries with XML tags plus an explicit data-not-instructions framing is the correct defensive design (indirect prompt injection, Domain 6). Relying on the model (A), naive keyword filtering (C) and temperature (D) are unreliable.
A. Pin a specific model snapshot.
B. Set `temperature: 0`.
C. Enable streaming.
D. Use adaptive thinking with `xhigh` effort.
E. Randomise `top_p` each run.
**Answer: A and B.** Pinning the model and using `temperature: 0` minimise variance for a fair comparison. Streaming (C) is a delivery mechanism; high-effort thinking (D) adds variability; randomising `top_p` (E) is the opposite of reproducible.
A. The server stores conversation state; send only the newest message.
B. Resend the full history each turn, appending the assistant's prior `content` blocks verbatim before the next user turn.
C. Concatenate all turns into one long user string.
D. Only the `system` prompt persists between calls.
**Answer: B.** State is client-side; you resend the whole history, appending assistant blocks verbatim (including `thinking`/`tool_use`). The server is stateless (A); flattening into one string (C) breaks roles; nothing persists server-side (D).
A. `output_config.format` with a JSON schema (or a `strict: true` tool schema).
B. Document `citations` enabled on the input document.
C. Setting `temperature: 0` only.
D. Raising `max_tokens`.
E. Forcing a tool via `tool_choice` on Fable 5.1.
**Answer: A and B.** Schema-constrained output guarantees the shape, and document citations return the source spans used. Temperature (C) and `max_tokens` (D) affect neither shape nor grounding; forcing a tool on Fable 5.1 (E) returns 400.
A. Lowering `max_tokens`.
B. Passing an idempotency key on the batch-create request so a retry is deduplicated.
C. Switching to synchronous calls.
D. Adding more exponential backoff.
**Answer: B.** An idempotency key makes the create safe to retry — it runs once. `max_tokens` (A) is unrelated; synchronous calls (C) lose the batch discount and do not dedupe; more backoff (D) does not prevent a duplicate create.
A. Pad the prefix to 16k tokens.
B. Move the timestamp out of the cached prefix (after the cache breakpoint) so the prefix is byte-identical across calls.
C. Raise the cache TTL to 1 hour.
D. Disable caching; it does not help here.
**Answer: B.** Any byte change in the prefix defeats caching; moving the per-request timestamp after the breakpoint restores hits. Padding (A) does not fix a changing prefix; a longer TTL (C) still needs identical bytes; disabling (D) forfeits a real saving once the prefix is stabilised.
A. Send more requests per minute since RPM is fine.
B. Cut input tokens per request (cache the shared prefix, trim context) and/or request a higher tier.
C. Raise `max_tokens` so fewer requests are needed.
D. Lower `temperature` to reduce token usage.
**Answer: B.** The binding axis is input-tokens-per-minute, so reduce input tokens or raise the tier. Sending more RPM (A) ignores the binding axis; raising `max_tokens` (C) increases OUTPUT tokens; temperature (D) does not change token counts.
A. The client constructor and authentication (`AnthropicBedrock`, AWS IAM/SigV4).
B. The model ID format (Bedrock-style identifiers).
C. The meaning of `stop_reason` values.
D. Whether `max_tokens` is required.
E. The basic shape of `messages`/`system`.
**Answer: A and B.** Only the client/auth and model ID format change; the request body and control-field semantics are portable. `stop_reason` meanings (C), the `max_tokens` requirement (D) and the `messages`/`system` shape (E) are unchanged.
A. A new synchronous client per request inside the event loop, no timeout.
B. The async client with a deliberate `timeout`, bounded SDK retries for transient errors, and a concurrency cap that respects rate limits.
C. Disable all retries and timeouts to maximise throughput.
D. Unbounded concurrency so every request fires at once.
**Answer: B.** An async client with a timeout, bounded retries, and a concurrency cap is the robust pattern. A per-request sync client with no timeout (A) blocks the loop; disabling safety nets (C) drops transient recovery; unbounded concurrency (D) causes 429 storms.
## Key takeaways
- The Messages API is a stateless HTTP JSON API; you resend history each turn and bill from `usage`.
- Drive control flow from `stop_reason` – never parse prose, never rely on iteration caps.
- `content` is an array of typed blocks; iterate and filter, do not assume `content[0].text`.
- Stream with SSE; tool args arrive as `input_json_delta`, final `stop_reason`/`usage` on `message_delta`.
- Prompt caching (stable prefix first, `cache_control`) cuts input cost to ~10% on hits; Batches give 50% off for latency-tolerant work.
- Retry only `429`/`500`/`529` with exponential backoff + jitter, honour `retry-after`, and log the request ID.
- `budget_tokens` is Haiku-4.5-only; current models use `thinking: {type: 'adaptive'}` with effort levels.
- Access via Bedrock/Vertex/Foundry when data or compliance requires it; the request shape is unchanged.
- Keep model IDs pinned in one place, prompts versioned, and secrets in env/secret manager – never in prompts or `CLAUDE.md`.
- Schema-constrained output (`output_config.format` / `strict: true`) guarantees shape, not values; validate semantics and enable document citations when grounding must be traceable.
- Reliability is layered: idempotency keys on writes, deliberate timeouts for long calls, bounded retries, and a concurrency cap to avoid self-inflicted `429`s.
- Diagnose by layer — a `JSONDecodeError` is a parsing/model-output problem (schema + validation-retry), a `429` is transport (backoff + `retry-after`); applying the wrong fix is the classic trap.
- Across Bedrock/Vertex the request body and `stop_reason` semantics are portable; only the client/auth and model ID format change.