D3 · Prompt Engineering and Structured Output

This domain is worth roughly 12 of 60 items and maps to the Structured Data Extraction and CI/CD scenarios. It treats a prompt as system design, not a text trick: stable structure for caching, explicit success criteria, clear content boundaries, and – above all – getting reliable machine-readable output out of the model and validating it. The most-tested decision is which structured-output route to use, including Fable 5.1’s restriction that forbids forced tool choice.

Learning objectives

By the end of this page you should be able to:

1. Treat a prompt as a versioned, modular system with a stable cacheable prefix and system/user role separation.
2. Write explicit success criteria and rubrics and use XML tags as content boundaries.
3. Select few-shot examples well and choose between chain-of-thought and extended thinking.
4. Use prefilling to steer output shape.
5. Choose the right structured-output route – output_config.format JSON schema, tool-use-as-schema, or strict tools – including the Fable 5.1 no-forced-tool-choice rule.
6. Design schemas (enums, required, nullable, descriptions) and build a validation-retry loop with error feedback and defensive parsing.
7. Build extraction pipelines for heterogeneous documents with per-type metrics (anti-pattern 10) and handle refusal and max_tokens stop reasons.
8. Design evaluator-optimizer prompts and avoid same-session self-review (anti-pattern 9).

3.1 The prompt as system design

A production prompt is an engineered artefact with parts that change at different rates. Order them so the stable content sits first and can be cached.

[ system role ]  stable: role, rules, output contract, tools  ← cache_control here
        +
[ user role ]    variable: the specific task, the document, the question

• System vs user roles. Put durable instructions, the output contract and constraints in the system prompt; put the specific task and data in user turns.
• Stable prefix for caching. Cache reads cost ~0.1× base input; put the system prompt, tool definitions and any long reference documents first and mark the last stable block with cache_control: {"type": "ephemeral"}. Minimum cacheable prefix ~1024 tokens (2048 on Haiku).
• Modular assembly and versioning. Build prompts from composable blocks (role, rules, schema, examples) and version them so you can evaluate and roll back. Treat prompt changes like code changes.

system = [
    {"type": "text", "text": ROLE_AND_RULES},                       # stable
    {"type": "text", "text": OUTPUT_CONTRACT},                      # stable
    {"type": "text", "text": REFERENCE_DOC,
     "cache_control": {"type": "ephemeral"}},                       # stable, cache boundary
]
messages = [{"role": "user", "content": task_specific_input}]        # variable

3.2 Explicit success criteria and rubrics

Vague instructions produce vague output. State what good looks like – measurable criteria and, for judged tasks, a rubric.

Rubrics also drive evaluator-optimizer loops (§3.10) and evals: the evaluator scores against the same explicit criteria the generator was given.

3.3 XML tags as content boundaries

Claude is trained to respect XML-style tags. Use them to separate instructions from data and to delimit output regions – this reduces injection risk and makes parsing deterministic.

<instructions>Extract the invoice total. Return only the value.</instructions>
<document>
{ untrusted document text here }
</document>

Wrapping untrusted content in a clearly named tag both improves accuracy and signals to the model that the enclosed text is data, not instructions – a first-line defence against indirect prompt injection.

3.4 Few-shot example selection

Few-shot examples teach format and edge-case handling. Select them deliberately:

• Representative of the real distribution, including the hard/edge cases you care about.
• Consistent in format – the model mimics the shape, so any inconsistency propagates.
• Diverse enough to cover the classes, but not so many that they bloat context (and cost).
• Correct – a wrong example is worse than none.

For extraction, one example per document type usually beats ten examples of one type.

3.5 Chain-of-thought vs extended thinking

On current models thinking is adaptive; budget_tokens is removed on Fable 5.x / Opus 5 / Sonnet 5 (400 error) and only Haiku 4.5 still uses budget_tokens. Effort levels are low | medium | high (default) | xhigh – use xhigh for the hardest coding/agentic work on Opus 5 / Fable 5.1. On Fable 5.1 thinking is always on.

3.6 Prefilling

Prefill the start of the assistant turn to constrain the shape of the output – force JSON, skip preamble, or lock a format.

messages = [
    {"role": "user", "content": "Return the extracted fields."},
    {"role": "assistant", "content": "{"},        # prefill: forces the model to continue JSON
]

Prefilling is a lightweight steer, not a guarantee. For strong guarantees use structured outputs or strict tools (next). Note: prefilling interacts poorly with extended thinking on models where thinking must come first – prefer structured outputs there.

3.7 Structured output: the three routes

This is the highest-value decision table in the domain.

# Route 1: structured outputs via output_config.format
resp = client.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": f"<document>{doc}</document> Extract fields."}],
    output_config={"format": {"type": "json_schema", "schema": INVOICE_SCHEMA}},
)

# Route 3: strict tool as a schema (works on Fable 5.1 with tool_choice: auto)
tools = [{
    "name": "record_invoice",
    "description": "Record the extracted invoice fields.",
    "strict": True,
    "input_schema": INVOICE_SCHEMA,
}]
resp = client.messages.create(
    model="claude-fable-5-1",
    max_tokens=1024,
    tools=tools,
    tool_choice={"type": "auto"},   # NOT {"type":"tool"} or "any" on Fable 5.1
    messages=[{"role": "user", "content": prompt}],
)

3.8 Schema design

A good schema is self-documenting and constraining.

{
  "type": "object",
  "properties": {
    "invoice_number": { "type": "string", "description": "The vendor invoice ID as printed." },
    "total": { "type": "number", "description": "Grand total in the invoice currency." },
    "currency": { "type": "string", "enum": ["USD", "EUR", "GBP"] },
    "due_date": { "type": ["string", "null"], "description": "ISO 8601 date, or null if absent." },
    "line_items": { "type": "array", "items": { "type": "object" } }
  },
  "required": ["invoice_number", "total", "currency"],
  "additionalProperties": false
}

• enum constrains a field to known values (prevents free-text drift).
• required forces presence; omit optional fields from it.
• Nullable via "type": ["string", "null"] – model the “absent” case explicitly rather than inviting a hallucinated value.
• description on every field – the model reads these; they are the cheapest accuracy lever.
• additionalProperties: false keeps output tight.

3.9 Validation-retry loop and defensive parsing

Even with schemas, validate and be ready to retry with specific error feedback.

import json, jsonschema

def extract_with_retry(client, prompt, schema, max_attempts=3):
    messages = [{"role": "user", "content": prompt}]
    for attempt in range(max_attempts):
        resp = client.messages.create(
            model="claude-opus-5", max_tokens=1024, messages=messages,
            output_config={"format": {"type": "json_schema", "schema": schema}},
        )
        if resp.stop_reason == "refusal":
            return {"status": "refused"}                 # do not retry blindly
        if resp.stop_reason == "max_tokens":
            raise OutputTruncated()                      # raise max_tokens and retry
        text = "".join(b.text for b in resp.content if b.type == "text")
        try:
            data = json.loads(text)                      # defensive parse
            jsonschema.validate(data, schema)
            return {"status": "ok", "data": data}
        except (json.JSONDecodeError, jsonschema.ValidationError) as e:
            messages.append({"role": "assistant", "content": text})
            messages.append({"role": "user",
                             "content": f"That output failed validation: {e}. "
                                        f"Return only valid JSON matching the schema."})
    return {"status": "failed_validation"}

Key points: feed the specific error back so the model self-corrects; parse defensively (never eval); treat refusal and max_tokens as distinct outcomes, not validation failures.

3.10 Extraction pipelines for heterogeneous documents

When you extract from many document types (invoices, contracts, receipts), a single aggregate accuracy number hides a type that is failing badly.

                 ┌─► classify type ─┬─► invoice schema  ─► validate ─► metrics[invoice]
Document ─► route ┤                 ├─► contract schema ─► validate ─► metrics[contract]
                 └─────────────────┴─► receipt schema  ─► validate ─► metrics[receipt]
                          per-type accuracy, not one aggregate number

Design: route by type → apply the type-specific schema → validate → record per-type metrics. Report and alert per type.

3.11 Handling refusal and max_tokens

Treating max_tokens output as a complete, parseable result is a silent-failure trap.

3.12 Evaluator-optimizer prompts and independence

For a generate-critique-refine loop, give the evaluator the explicit rubric and run it as an independent context – a fresh session, ideally a different model. Reusing the generating conversation to grade itself keeps the reasoning bias that produced the flaw.

3.13 Schema design patterns for hard extraction cases

Beyond enums and nullable types, several schema patterns decide whether extraction is reliable on messy documents.

{
  "type": "object",
  "oneOf": [
    { "properties": { "doc_type": { "const": "invoice" }, "total": { "type": "number" } },
      "required": ["doc_type", "total"] },
    { "properties": { "doc_type": { "const": "contract" }, "term_months": { "type": "integer" } },
      "required": ["doc_type", "term_months"] }
  ]
}

3.14 A robust validation-retry harness (with schema-error feedback)

The exam-correct loop treats refusal and max_tokens as distinct outcomes, feeds the specific validation error back, and never parses unsafely.

import json, jsonschema
from jsonschema import Draft202012Validator

def extract(client, prompt, schema, model="claude-opus-5", max_attempts=3):
    messages = [{"role": "user", "content": prompt}]
    for attempt in range(max_attempts):
        resp = client.messages.create(
            model=model, max_tokens=2048, messages=messages,
            output_config={"format": {"type": "json_schema", "schema": schema}},
        )
        if resp.stop_reason == "refusal":
            return {"status": "refused"}                       # safety stop — do not bypass
        if resp.stop_reason == "max_tokens":
            return {"status": "truncated", "retry": "raise_limit_or_chunk"}  # not complete
        text = "".join(b.text for b in resp.content if b.type == "text")
        try:
            data = json.loads(text)                             # never eval()
            errors = sorted(Draft202012Validator(schema).iter_errors(data),
                            key=lambda e: e.path)
            if not errors:
                return {"status": "ok", "data": data}
            detail = "; ".join(f"{list(e.path)}: {e.message}" for e in errors[:5])
        except json.JSONDecodeError as e:
            detail = f"invalid JSON at pos {e.pos}: {e.msg}"
        messages += [
            {"role": "assistant", "content": text},
            {"role": "user", "content": f"Validation failed: {detail}. "
                                        f"Return ONLY valid JSON matching the schema."},
        ]
    return {"status": "failed_validation"}

import Ajv from 'ajv';
const ajv = new Ajv({ allErrors: true });

async function extract(client, prompt, schema, model = 'claude-opus-5', maxAttempts = 3) {
  const validate = ajv.compile(schema);
  const messages: any[] = [{ role: 'user', content: prompt }];
  for (let i = 0; i < maxAttempts; i++) {
    const resp = await client.messages.create({
      model, max_tokens: 2048, messages,
      output_config: { format: { type: 'json_schema', schema } },
    });
    if (resp.stop_reason === 'refusal') return { status: 'refused' };
    if (resp.stop_reason === 'max_tokens') return { status: 'truncated' }; // not complete
    const text = resp.content.filter(b => b.type === 'text').map(b => b.text).join('');
    try {
      const data = JSON.parse(text);                    // never eval / Function
      if (validate(data)) return { status: 'ok', data };
      const detail = (validate.errors ?? []).slice(0, 5)
        .map(e => `${e.instancePath} ${e.message}`).join('; ');
      messages.push({ role: 'assistant', content: text },
        { role: 'user', content: `Validation failed: ${detail}. Return ONLY valid JSON.` });
    } catch (e) {
      messages.push({ role: 'assistant', content: text },
        { role: 'user', content: `Invalid JSON: ${e}. Return ONLY valid JSON.` });
    }
  }
  return { status: 'failed_validation' };
}

3.15 Structured-output cost and token arithmetic

Structured output is not free of token cost, and schema verbosity shows up on the bill. A quick model:

Prompt = system(1,500) + tools/schema(800) + document(6,000) = 8,300 input tokens
Output = ~400 tokens of JSON

On Opus 5 ($5/M in, $25/M out):
  input  = 8,300 × $5  / 1e6 = $0.0415
  output =   400 × $25 / 1e6 = $0.0100
  per doc                     ≈ $0.0515

Cache the stable 2,300-token prefix (system + schema), read ≈ 0.1×:
  cached prefix = 2,300 × $0.5 / 1e6 = $0.00115 (vs $0.0115 uncached)
  → saves ≈ $0.0104 per doc; over 1M docs ≈ $10,400 saved.

Batch API (latency-tolerant) halves the remaining cost → ~$0.026/doc.

Common misconceptions

Scenario walkthrough — an extraction pipeline that looks fine but ships bad contracts

Situation. A pipeline extracts fields from invoices, contracts and receipts on Fable 5.1 and writes to a database. It reports 96% overall accuracy, yet the legal team keeps finding wrong contract term lengths. The engineers force tool_choice onto a record tool and see intermittent 400s; when a big contract is processed, the JSON sometimes ends mid-array and the code writes the partial object. A proposal is to “grade outputs by asking the model, in the same chat, if it is confident.”

Expert reasoning trace.

1. Fix the 400s first. Fable 5.1 forbids forced tool_choice. Switch to structured outputs (output_config.format) or a strict: true tool with tool_choice: "auto" plus an instruction. Reject “retry with backoff” (a 400 is deterministic) and "any" (also blocked).

2. Fix the silent truncation. JSON ending mid-array with stop_reason == max_tokens is truncation, not a result. Raise max_tokens, or chunk the contract (or bound line_items with maxItems), then retry. Never write the partial object.

3. Fix the metric. 96% overall masks a failing type (#10). Report per-document-type accuracy, alert on contracts, and gate release on the worst type. Reject “bigger sample” and “average more runs” — both still aggregate.

4. Fix the evaluation design. Same-session self-confidence is #9 and also self-reported confidence territory. Use an independent judge (fresh session, ideally a different model) scoring against a written rubric.

5. Harden the schema. Contracts need a discriminated union branch with term_months as a typed integer and a nullable + found flag for optional clauses so absence is not hallucinated.

Exam-correct decision: structured outputs / strict tools (not forced choice), truncation handling by raising/chunking, per-type metrics gating on the worst type, an independent evaluator, and a discriminated-union schema. Each rejected option is a named trap (Fable forced choice, silent truncation, aggregate masking #10, same-session #9).

Exam traps in this domain

Practice questions

Key takeaways

• Design prompts as versioned, modular systems: stable content first (system role, tools, docs) with cache_control; variable task last.
• State explicit success criteria and rubrics; use XML tags to separate untrusted data from instructions.
• Pick structured output deliberately: output_config.format for schema guarantees, tool-use-as-schema/strict for record emission; never force tool_choice on Fable 5.1.
• Design schemas with enums, required, nullable types, descriptions, discriminated unions and bounded arrays; model absence with nullable + a found flag.
• Build a validation-retry loop that feeds the specific schema error back and parses safely (never eval); treat refusal and max_tokens as distinct outcomes.
• Slice extraction metrics per document type and gate on the worst; never trust one aggregate number.
• Control reasoning with adaptive thinking + effort (Opus/Sonnet/Fable); only Haiku 4.5 uses budget_tokens.
• Run evaluators as independent contexts; on Fable 5.1 keep the harness append-only and freeze system/tools.
• For bulk extraction, cache the stable schema/system prefix and use the Batch API (50% off) — the cost levers, not a bigger model.