aifromzero.dev

Capstone: Ship Relay, a Production-Grade GenAI Support Agent (AWS GenAI Developer Pro, Module 15)

Module 15 of 16 16 min read D1 · 31%D2 · 26%D3 · 20%D4 · 12%D5 · 11% Lab code ↗

This is Module 15 of AWS GenAI Pro Mastery, a free 16-module course that takes you from your first Amazon Bedrock call to AWS-certified.

Friday, 6 p.m. CloudCart’s head of support drops the message in the channel: Relay goes live Monday. You open the repo. Fourteen modules, every test green: triage emits clean JSON, the Knowledge Base cites its sources, the agent calls its tools, the guardrail blocks the injection, the dashboard shows the $/ticket. By any local measure, it works.

Then the cold thought arrives. Nobody has ever run one real customer ticket through the whole thing end to end and read the number that came out. Worse: CloudCart’s webhook retries on timeout, like every webhook does. If it delivers the same refund ticket twice, does Relay refund twice? And if the agent loops on a failing tool, who stops it?

Here is the pain this module fixes: having built every piece is not having a system. A system survives the real world — a duplicated webhook, a runaway loop, a cost spike — and you can destroy and rebuild it on demand. By the end of this capstone, Relay is one shippable thing you can prove, then tear back to ~$0/month.

In this module

You’ll do: assemble, harden, review, demonstrate, ship, and tear down.

You’ll build: Relay v1.0 — one cohesive, hardened system, reviewed against the Well-Architected Generative AI Lens, proven on a 20-ticket costed run, tagged and shipped — then fully destroyed back to ~$0/month.

Exam domains covered: all five domains — this is the synthesis module.

  • D1 — Foundation Model Integration, Data Management, and Compliance — 31%
  • D2 — Implementation and Integration — 26%
  • D3 — AI Safety, Security, and Governance — 20%
  • D4 — Operational Efficiency and Optimization for GenAI Applications — 12%
  • D5 — Testing, Validation, and Troubleshooting — 11%

The capstone exercises all five; it re-teaches none. The only new skills here are 1.1.1 (solution design) and 1.1.3 (the Generative AI Lens). Everything else is consolidation of Modules 2–14.

Prerequisites: Modules 1–14 complete, an AWS_PROFILE configured for us-east-1, and the Module 1 budget alarm still in place.

Where you are in the build

You are 15 modules into a 16-module build. The pieces all exist; this is the module that turns them into something you can ship.

  • ✅ M1–M14 — Relay triages, answers from a Knowledge Base with citations, acts through tools and an agent, hands off to a Billing specialist with a human-in-the-loop refund gate, is guarded and PII-redacted, ships behind an API with CI/CD, knows what every ticket costs, gates its own deploys on a golden set, and is observed by the relay-ops dashboard. Fourteen pieces, built and tested in isolation — never assembled, hardened, or demonstrated end to end.
  • 👉 M15 — you are here. Assemble the fourteen pieces, harden them against the traffic that misbehaves, review them pillar by pillar, demonstrate them on a 20-ticket costed run, tag v1.0 — then verify a teardown back to ~$0/month.
  • ⬜ M16 — the exam. Module 16 doesn’t build Relay — it gets you certified.

The assembly: one system, fourteen modules

This is not a new concept. It is a map. Solution design — creating an architectural design aligned with business needs and technical constraints — is skill 1.1.1, and the assembled Relay is that design. So the first job of the capstone is to redraw the target architecture, tag every block with the module that built it, and ask the only question that matters at ship time: do these choices hold together?

flowchart TB
    subgraph Edge["Front door — M11 (API Gateway + CDK + CodePipeline)"]
        POST["POST /tickets → 202"]
        GET["GET /tickets/{id}"]
        APPR["POST /tickets/{id}/approve"]
        SQS["SQS buffer"]
    end
    subgraph Pipe["Processing — the frozen run_relay seam (M8→M11)"]
        INTAKE["intake + PII redact<br/>M6 / M10"]
        TRIAGE["triage classifier · M2<br/>(standalone — not on the autonomous run_relay seam)"]
        AGENT["Strands ReAct agent<br/>M7 (+ MCP tools)"]
        SPEC["Billing specialist + HITL<br/>M8 (AgentCore Runtime + Memory)"]
    end
    subgraph Data["Knowledge & state"]
        KB["Knowledge Base relay-kb<br/>on S3 Vectors · M4 / M5"]
        DDB["DynamoDB relay-orders / relay-tickets<br/>+ semantic cache · M7 / M12"]
    end
    subgraph Cross["Cross-cutting"]
        GUARD["relay-guardrail<br/>M9"]
        LLM["converse() — single call site<br/>M3 (tiers + retries + backoff)"]
        BUS["relay-events bus<br/>M11"]
        OPS["relay-ops dashboard + 4 alarms<br/>M14"]
        EVAL["golden-set evals + gate<br/>M13"]
    end
    POST --> SQS --> INTAKE --> AGENT
    INTAKE -.->|"M2 classifier, off the autonomous seam"| TRIAGE
    AGENT --> SPEC
    AGENT -->|search_kb| KB
    AGENT -->|lookup_order / create_ticket| DDB
    AGENT --> LLM
    GUARD -.->|"standalone ApplyGuardrail + converse() hook · M9<br/>(not on the agent's ReAct path)"| LLM
    SPEC -->|approval_required / escalation| BUS
    GET -.reads.-> DDB
    APPR -.approve refund.-> SPEC
    OPS -.observes.-> Pipe
    EVAL -.gates deploy.-> Edge

The choices line up against CloudCart’s constraints. Cost stays low: Amazon S3 Vectors backs the RAG index instead of OpenSearch Serverless (~$0 idle versus ~$174/month, as of June 2026), and DynamoDB and AgentCore Runtime are on-demand with free idle. Integration stays disciplined: every generation call goes through one place — the single converse() call site in relay/llm.py — so model IDs, retries, and routing live in exactly one file. Throughput stays decoupled: the API returns 202 and an SQS buffer feeds the worker, so a burst queues instead of failing. Sensitive actions stay safe: refunds pause on a human-in-the-loop gate; nothing executes without an explicit approval. That alignment of components to constraints is the whole of skill 1.1.1 — you can read it straight off the diagram.

The capstone consumes every contract those fourteen modules froze, changing none of them — the converse() signature is byte-identical from Module 3 through here, and the TicketRecord status machine still has exactly its seven states:

converse(messages, *, tier="auto", stream=False, **params)   # byte-identical M3→M15

# TicketRecord.status — frozen M7, fully exercised by the run below
status: Literal["received", "triaged", "awaiting_approval",
                "answered", "escalated", "closed", "failed"]

Hardening for production: idempotency, timeouts, quotas

The system answers tickets. That is not the same as being production-ready — ready to take real traffic, including the traffic that misbehaves. Production-readiness is resilience added around the contracts, changing none of them. Three failure modes matter most, and Relay was missing a guard for each.

A duplicate delivery: CloudCart’s webhook retries, so the same ticket arrives twice. An idempotency key — a value that lets a repeated request produce the same result as a single one — fixes it. Relay’s front-door received write becomes a conditional attribute_not_exists(ticket_id) PutItem, so the second delivery is a no-op, never a second pipeline or a second refund.

A runaway agent: a tool keeps failing and the ReAct loop keeps retrying. Timeouts — a wall-clock and a step cap — bound it. The bedrock-runtime client carries a read/connect timeout and the loop carries MAX_ITERATIONS=6, so a stuck run ends failed cleanly instead of hanging and burning tokens.

A traffic burst: more tickets arrive than Bedrock will serve at once. Quotas / throttling — limits on concurrency that smooth load instead of dropping it — absorb it. Lambda reserved concurrency plus the SQS buffer plus the existing converse() exponential backoff turn a spike into a queue, not an outage.

The full hardening map — every failure mode to the resilience that catches it — is the lab’s required table:

Failure modeHardeningWhere (module)
A CloudCart webhook delivers the same ticket twiceIdempotency key on the front-door received write — a conditional attribute_not_exists(ticket_id) PutItem; a duplicate is an idempotent no-op (202 duplicate:true), never a second pipeline or refundmcp_server/store.py, relay/api/post_handler.py (M15)
The agent loops on a failing toolTimeout + stop conditions — AGENT_TIMEOUT_S=60, MAX_ITERATIONS=6, bedrock-runtime read/connect timeouts; the run ends failed, not hungrelay/agent.py (M7, applied here)
A burst of tickets exhausts the Bedrock quotaQuotas / throttling — Lambda reserved concurrency + SQS buffer; converse() exponential backoff + jitter on throttlingcdk/, relay/config.py, relay/llm.py (M11/M3)
A tool (DynamoDB) errors mid-runPartial failure handling — the tool returns a model-readable error (no crash); a poison SQS job redrives to a DLQ after RELAY_QUEUE_MAX_RECEIVE=3mcp_server/store.py, relay/api/worker_handler.py (M7/M11)
A refund is proposed but not approvedStays awaiting_approval — nothing executes without POST /tickets/{id}/approverelay/agent.py, relay/approve.py (M8)
A cost spike with flat trafficThe M1 budget alarm + the relay-ops cost-anomaly alarm (learned band)M1 / M14

Notice the seam called partial failure — the case where one step of a multi-step run fails and the rest must degrade, not crash. A failing tool hands the model a readable error and the run lands on the failed terminal status; a poison message redrives to a dead-letter queue after three receives. Nothing in this table touches a Pydantic schema, the converse() signature, or a resource name. That is the point.

⚠️ Common misconception: shipping means adding the last feature. It does not. Shipping is hardening (idempotency, timeouts, quotas, partial-failure handling), measuring (the costed run), reviewing (the Lens), and being able to destroy and rebuild on demand. None of that is a new feature. Relay shipped with no new surface area — it gained resilience, not features.

The Well-Architected Generative AI Lens review

The last skill the capstone teaches is 1.1.3: creating standardized components for consistent implementation, anchored on the AWS Well-Architected Generative AI Lens. The Lens is the official AWS Generative AI Lens — published both as a whitepaper and as a custom lens you load into the AWS Well-Architected Tool (the exam guide names it “AWS WA Tool Generative AI Lens” — the exact term you will see on the test) — and it extends the AWS Well-Architected Framework, the six-pillar discipline AWS uses to review any workload, with the concerns specific to generative AI: model selection, RAG grounding, responsible AI, human-in-the-loop, and evaluation. It exists to standardize the review of a GenAI architecture so two engineers reach the same verdict, and so the gaps surface before the go-live, not in the post-incident review.

It reads as a checklist, not prose. The Lens organizes the six pillars — Operational Excellence, Security, Reliability, Performance Efficiency, Cost Optimization, and Sustainability — across the generative-AI lifecycle, and for each best practice you mark the concrete control already built and the gap that remains. Relay’s review (docs/genai-lens-review.md) maps every pillar to a module:

PillarRelay control (module)Gap → “In production”
Operational Excellencerelay-ops dashboard + 4 alarms + a proven runbook (M14); golden-set canary (M13/M14)A paging on-call rotation; scheduled canary cadence
Securityrelay-guardrail + prompt-attack filter (M9); Comprehend PII redaction before any FM call (M10); least-privilege IAM, zero wildcards (M10)KMS CMKs; a red-team pass beyond the 12-attack suite
Reliabilityconverse() retries + cross-Region fallback (M3); idempotency key (M15); agent timeouts (M7); SQS DLQ (M11)Multi-Region active/active
Performance EfficiencyTier router fast/smart/auto (M3); prompt + semantic caching (M12); p95 widget (M14)Load testing under realistic concurrency
Cost Optimizationcost_cents metering (M12); S3 Vectors over OpenSearch (M4/M5); Flex/batch on non-interactive paths only (M12); exhaustive teardown (M15)Continuous FinOps on $/ticket
SustainabilitySmallest sufficient model per task (M3); caching skips repeated generation (M12); decommission to ~$0 (M15)A measured tokens-per-ticket budget

Every pillar passes with a built control. The ⚠️ rows — the gaps — are not failures; they are the agenda for the next quarter, exactly the “In production” frontier below. That is what a Lens review is for: it tells you what a reviewer would ask for next, in a form you can hand to one.

Build it: assemble, demonstrate, ship, tear down

The lab is the heart of this module. Copy the cumulative state from module-14/ — the whole relay/ package, evals/, observability/, the CDK app, docs/runbook.md — and add only the capstone increment: demo_capstone.py, docs/genai-lens-review.md, the hardening by addition, and the v1.0 README. uv sync installs no new dependency. This lab cost me $0.41 on June 2026 prices — well under the $3 budget — with the headline 20-ticket run metering 7.46¢ of real Bedrock tokens.

At the end, uv run python demo_capstone.py prints one costed recap proving the fourteen pieces are now one system.

Harden by addition — the idempotency key

The front-door write becomes conditional. A first delivery succeeds; a second of the same ticket_id raises IdempotentReplay instead of clobbering the row or starting a second run. No signature changes — it is a new sibling of create_ticket:

# mcp_server/store.py — Module 15 hardening (idempotency), by addition.
table = _tickets_table(resource)
try:
    table.put_item(
        Item=item,
        # The idempotency gate: only write when no row for this ticket_id exists.
        ConditionExpression=f"attribute_not_exists({config.TICKETS_KEY})",
    )
except ClientError as err:
    if err.response["Error"]["Code"] == "ConditionalCheckFailedException":
        raise IdempotentReplay(
            f"Ticket {record.ticket_id!r} already exists — duplicate delivery "
            "(idempotent replay). The first record stands; no second pipeline or action."
        ) from err
    raise

The handler treats the replay as a no-op success, returning the same ticket_id and not enqueuing a second job:

# relay/api/post_handler.py — a duplicate webhook is a replay, not a new ticket.
try:
    write_received(ticket, persist=persist)
except IdempotentReplay:
    print(f"[post_handler] duplicate delivery of {ticket.ticket_id!r}; idempotent no-op.")
    return common.response(
        202, {"ticket_id": ticket.ticket_id, "status": "received", "duplicate": True}
    )

The smoke test proves it: two submissions of the same ticket leave exactly one TicketRecord, with no second enqueue and no second refund.

Bound the agent — the timeout

The timeout and step cap were defined back in Module 7; the capstone applies them, it does not re-specify them. The Strands BedrockModel carries the wall-clock guard through its boto_client_config, so a stuck model call cannot outlast the loop:

# relay/agent.py (M7) — applied here. The Strands BedrockModel carries the
# wall-clock guardrail via boto_client_config; the ReAct loop carries the step cap.
MAX_ITERATIONS = 6
AGENT_TIMEOUT_S = 60
return BedrockModel(
    model_id=config.tier_profile(AGENT_TIER),   # the model ID lives only in config
    region_name=config.REGION,
    boto_client_config=BotoConfig(
        read_timeout=AGENT_TIMEOUT_S,
        connect_timeout=10,
        retries={"max_attempts": 2, "mode": "standard"},
    ),
)
# ... agent runs with limits={"turns": MAX_ITERATIONS}; a runaway ends `failed`, not hung.

Demonstrate — the 20-ticket costed run

The demo drives 20 varied tickets end to end: 16 nominal across every intent, 2 adversarial reusing the Module 9 attacks, and 2 multimodal reusing the Module 6 payment-error screenshot. For each one it prints the outcome — by status, a cited answer, an action, an escalation, or awaiting_approval. (The autonomous run_relay seam lets the agent reason directly; it does not run the Module 2 triage classifier, so the recap reports status, not a triaged intent.) It runs through the exact frozen run_relay seam the deployed worker invokes: the --api-url path hits POST /tickets → poll GET on a deployed stage, and the default local path drives the same seam wrapped in a cost meter, so the recap is reproducible without standing up the whole stack first. (One caveat: on the --api-url path the cost reflects the deployed worker’s CostMeter, which sums converse() calls — and the Strands agent bypasses converse(), so agent tokens are under-counted there; the local path wraps the run in the agent usage meter that captures them. Threading that meter into the deployed worker is a documented follow-up.) No new model client, no new generation path — it consumes run_relay.

def print_recap(outcomes, recap):
    print("RECAP")
    print(f"  tickets               : {recap['tickets']}")
    print(f"  by status             : {recap['by_status']}")
    print(f"  by category           : {recap['by_category']}")
    print(f"  escalation rate       : {recap['escalation_rate']:.1%}")
    print(f"  total $/ticket        : {recap['total_cost_usd']:.4f} USD "
          f"({recap['total_cost_cents']:.3f}¢ over {recap['tickets']} tickets)")
    print(f"  p95 latency           : {recap['p95_latency_ms']:.0f} ms")
    eg = recap.get("eval_grounding")
    print(f"  golden-set grounding  : {eg if eg is not None else 'skipped'}"
          f"{'  (floor ' + str(config.GROUNDING_THRESHOLD) + ')' if eg is not None else ''}")

The real run, live against Bedrock + relay-kb + relay-guardrail + the Strands agent + MCP tools + AgentCore Memory + DynamoDB:

RELAY v1.0 — CAPSTONE RUN (20 tickets, end-to-end)
RECAP
  tickets               : 20
  by status             : {'answered': 13, 'failed': 5, 'awaiting_approval': 2}
  by category           : {'nominal': 16, 'adversarial': 2, 'multimodal': 2}
  escalation rate       : 0.0%
  awaiting approval     : 2
  total cost (run)      : 0.0746 USD (7.457¢ over 20 tickets)
  cost per ticket       : 0.3729¢
  p95 latency           : 7327 ms
  golden-set grounding  : 0.963  (floor 0.8)

Read the recap as evidence, not decoration. The two refunds parked on the human-in-the-loop gate (awaiting_approval) — nothing executed without an approval. Escalation read 0.0% this run only because both refunds parked on that gate instead of escalating; escalated is a live terminal status in the state machine, it just did not fire this time, and both awaiting_approval outcomes are valid. The two adversarial tickets were handled safely: no PII exfiltration, the system prompt and the IAM tool boundary held. (The standalone relay-guardrail from Module 9 guards the converse()-path answers and the run_attacks.py suite; it does not sit on the Strands agent’s ReAct loop, which is why the agent path leans on the system prompt and the IAM tool boundary — wiring the guardrail onto the agent’s request path is the named next step from Module 9.) The two vision tickets answered from the screenshot. And the five failed are the hardening working — the MAX_ITERATIONS=6 turn cap ending a KB-search runaway cleanly with status failed, not a crash. Be precise about what 5/20 means, though: failed here is a guardrail/turn-cap terminal status, not silently-dropped customer work. The two adversarial probes are meant to terminate without a normal answer; the rest are legitimate but KB-heavy tickets the agent could not resolve inside six turns. That is a tuning signal, not a defect — in production you would raise MAX_ITERATIONS for genuinely retrieval-heavy intents so a hard-but-valid ticket gets more turns before the cap clips it, while keeping the cap as the safety net against an actual runaway. The cost_cents is the real metered smart-tier token cost — the Module 12 instrumentation, consumed as-is, captured by a non-invasive usage meter since the Strands agent bypasses converse(). The grounding 0.963 is a re-run of the Module 13 golden set through the same run_evals.py contract, comfortably above the 0.8 floor.

That evals contract is consumed field-for-field, never modified:

uv run python evals/run_evals.py --out evals/results/run-<name>.json
→ { run_name, config, scores: [ {id, triage_ok, grounding, coverage, citations} ],
    aggregate, cost_cents }
Gate: fail if aggregate.grounding < 0.8 or a regression > 5 pts vs baseline.

Release, then tear down to ~$0

Tag the contract deliverable and finalize the README — the pitch, the architecture, the run numbers, the teardown command — a portfolio-ready repo, not a tutorial:

git tag v1.0                      # the contract deliverable (06 §3)
uv run python teardown.py         # EXHAUSTIVE — incl. AgentCore long-term Memory; ~$0/month

The teardown is the capstone’s whole point. teardown.py is idempotent and exhaustive — it removes every resource the build created:

  • the relay-ops dashboard + 4 alarms + SNS topic; invocation logging disabled and log groups purged;
  • the cache table; the MCP Lambda + Function URL + bounded role;
  • the AgentCore long-term Memory purged — the only monthly idle item, ~$0.75/1K records/month;
  • the guardrail; the KB + its role; the eval/batch roles + S3 artifacts; the agent tables;
  • and — when deployed — the CDK API + pipeline stacks.

The Module 1 $5 budget alarm is kept on purpose. Then verify, on Cost Explorer or a resource inventory, that nothing is idle-billed: account back to ~$0/month. Live smoke confirmed the assembled system one last time — RELAY_LIVE_TESTS=1 uv run pytest -m live → 14 passed, 1 skipped, including one real ticket end to end; offline, 350 passed, 15 skipped.

Try it yourself: (1) add a 21st ticket of your own — a partial-refund edge case — to demo_capstone.py and explain where it lands in the recap; (2) add one row to docs/genai-lens-review.md for the gap you would close first in production, and say why — the cost of being wrong, or the blast radius, should drive the order.

In production

The Lens review already named the frontier; here it is, one line each, as the work beyond this course. Multi-Region real active/active, not just cross-Region inference for the model call. A formal SLA and error budget so quality has a contract, not a vibe. An on-call rotation with paging and a runbook that grows with every incident. Load testing under realistic concurrency before you trust the auto-scaling floors. A blue/green or canary model rollout so a model swap is reversible mid-flight. A third-party security and responsible-AI audit beyond the 12-attack suite and the fairness rubric. Compliance work — SOC 2, data residency, KMS customer-managed keys on the data at rest. And continuous FinOps on the $/ticket, because the cost number drifts the moment traffic patterns change. Each is a ⚠️ row in the review, deliberately out of scope here — and each is the next thing a reviewer would ask for, the difference between a system that runs and one a company will stake its support queue on.

Exam corner

What the exam tests here. The capstone touches all five domains through one system. Task 1.1 owns the two skills it actually teaches — 1.1.1, architectural designs aligned with business needs and technical constraints, and 1.1.3, standardized components and the AWS Well-Architected Framework / Generative AI Lens. The rest is synthesis: the exam loves the gap between works in a demo and production-ready, and it tests it from every angle — which control covers which pillar, where a cost regression hides, and which missing guard creates a financial risk.

Quiz.

  1. A company needs an internal support agent: low cost at idle, answers grounded in its own private docs, and any refund must be approved by a human. Which combination best fits these constraints?

    • A. OpenSearch Serverless for RAG, a DIY retrieval loop, and the agent auto-executing refunds for speed.
    • B. S3 Vectors-backed Knowledge Base for grounded answers, and a human-in-the-loop gate that parks refunds in awaiting_approval until approved.
    • C. A single frontier model on every ticket, no retrieval, and refunds logged for later review.
    • D. A provisioned-throughput fine-tuned model with refunds executed inside the same tool call.

  2. Reviewing the Security pillar of the Generative AI Lens, you must cover two risks: a customer pasting untrusted content into a ticket, and PII leaking into the model or logs. Which controls map to them?

    • A. IAM least-privilege for both.
    • B. A guardrail with a prompt-attack filter for untrusted content; Comprehend PII redaction at intake before any FM call for the PII leak.
    • C. A guardrail for both; PII handling is automatic.
    • D. Encryption at rest for both.

  3. Relay’s $/ticket doubles overnight with no increase in traffic. Where do you look first?

    • A. The monthly invoice.
    • B. A bug in the CDK stack — redeploy.
    • C. The relay-ops dashboard and the agent traces, for a tool loop inflating token usage.
    • D. The DynamoDB capacity mode.

  4. Moving all traffic to the smart tier improves answer quality but quadruples cost. Which mechanisms already in Relay preserve both quality and cost?

    • A. Switching every ticket to the fast tier.
    • B. The auto complexity router plus prompt and semantic caching plus per-task tiering.
    • C. Provisioned throughput on the smart model.
    • D. Running every ticket through the guardrail twice.

  5. A teammate says Relay is production-ready because the demo answers every ticket correctly. Which gap creates a direct financial risk?

    • A. The README is missing a diagram.
    • B. The reading time is unset.
    • C. A non-idempotent front door (a duplicated webhook double-refunds) or an idle-billed resource left after teardown (e.g. unpurged AgentCore long-term Memory).
    • D. The relay-ops dashboard is missing a chart title.

Answers. 1 — B. The constraints are low idle cost, private-doc grounding, and human approval on refunds; a Knowledge Base on S3 Vectors gives grounded answers at $0 idle, and the human-in-the-loop gate parks refunds until approved. A auto-executes a sensitive action and pays for idle OpenSearch Serverless ($174/month idle, versus ~$0 for S3 Vectors); C drops grounding; D over-provisions and skips approval. This is skill 1.1.1 — match components to constraints. 2 — B. Untrusted content is a guardrail / prompt-attack concern; a PII leak is solved by redaction at the intake edge, before any FM call. IAM (A) bounds permissions, not content or PII text; encryption (D) protects data at rest, not what reaches the model. The Lens maps each pillar risk to a specific control. 3 — C. A cost doubling with flat traffic points at a token-usage change, not infra — most often an agent tool loop — and the first signals are the dashboard and the agent traces. The invoice (A) is weeks late, the stack (B) and capacity mode (D) did not change. 4 — B. The router sends simple tickets to fast and hard ones to smart, caching skips repeated generation, and per-task tiering keeps quality where it is needed without paying smart-tier prices everywhere. A trades away quality, C pays for idle capacity, D doubles guardrail cost for nothing. 5 — C. A non-idempotent front door can refund a customer twice on a webhook retry, and an unpurged AgentCore long-term Memory keeps billing monthly after “shutdown” — both are money. The others are cosmetic. The exam tests exactly this demo ≠ production gap.

Traps to avoid.

  • Believing “works in a demo” means “production-ready.” What is missing is the hardening (idempotency, timeouts, quotas), continuous observability, and a verified teardown. The exam tests precisely this gap.
  • Forgetting idle cost at teardown. OpenSearch Serverless, provisioned throughput, or an unpurged AgentCore long-term Memory keep billing even when the app is “off.” An exhaustive teardown is a skill, and idle-cost discipline is part of Cost Optimization — a domain the exam scores (D4).
  • Treating the Generative AI Lens as an end-of-project audit. It is a pillar-by-pillar review you run continuously, surfacing gaps before the go-live — not a compliance stamp at the end.

Key takeaways

  • A system is not the sum of its pieces — the assembly is itself a deliverable, and skill 1.1.1 grades it.
  • Hardening is idempotency + timeouts + quotas + partial-failure handling — resilience around the contracts, not more features.
  • The Well-Architected Generative AI Lens is a pillar-by-pillar review that reveals concrete gaps, not a form you file.
  • “Works in a demo” is not production-ready — it is missing hardening, continuous observability, and a verified teardown.
  • The costed run turns “looks good” into proof: by-status counts, escalation rate, $/ticket, p95, and grounding, on real tickets.
  • A verified teardown to ~$0/month is a professional skill — and a financial guard the exam scores (purge the AgentCore long-term Memory).
  • You now have a portfolio project you can ship, destroy, and rebuild on demand — not a tutorial you followed once.

Want the full AIP-C01 question bank and the next module in your inbox? Subscribe here — it’s free, like everything in this series.

You’ve built it. Module 16 gets you certified: exam strategy, a weighted mock exam, and my debrief.

Links: Module 15 lab (tag v1.0) · ← Module 14 · Course index · Module 16 →

Where to go next

You have a shippable project. Here is how to extend it, study from it, and present it.

Extend the project. Add a Slack or Teams channel as a new intake source feeding the same POST /tickets. Add a second specialist beside the Billing one. Or, as theory for now, fine-tune a small Nova model for triage and A/B it against the prompt-based triage — a cost/quality trade-off to measure before switching.

Three directions are worth calling out because each is a real next step the blueprint names, not just a nice-to-have. Give it a human-usable front end (skill 2.5.2). The next layer up is the accessible-interface tier: stand up an AWS Amplify app for the support team and publish an OpenAPI contract over the POST /tickets, GET /tickets/{id}, and POST /tickets/{id}/approve routes Relay already exposes, so the UI and any future client are generated from the contract instead of hand-wired to it. Hand flow editing to non-engineers (skill 1.6.6). Once support leads want to reshape the triage-to-specialist sequence themselves, Amazon Bedrock Prompt Flows is the no-code orchestration alternative — they edit the chain in a visual canvas rather than filing a PR against run_relay. Accelerate the next iteration (skill 2.5.4). Reach for Amazon Q Developer to draft the second specialist, the Slack adapter, and their tests, so the iteration after v1.0 costs you a review pass instead of a blank file.

Study from it. Read the AIP-C01 exam guide for Task 1.1, work through AWS Skill Builder’s GenAI path, take the Official Practice Question Set, and read the Generative AI Lens end to end — Relay is a worked example of most of it.

Present it. When you walk an interviewer through Relay, lead with the pitch (a support agent that triages, grounds, acts, and escalates safely), then the architecture (the diagram above, every block tagged by module). Then show the real numbers from the run — 7.46¢ over 20 tickets, p95 7327 ms, grounding 0.963 — and close with the line that separates you from everyone who only ever prototypes: I can destroy it and rebuild it on demand. This is the project that proves you can ship, not just prototype.

References