Data Pipelines for FMs: Validation, Multimodal Input, and Document Processing (AWS GenAI Developer Pro, Module 6)

This is Module 6 of AWS GenAI Pro Mastery, a free 16-module course that takes you from your first Amazon Bedrock call to AWS-certified.

A real CloudCart support email lands in Relay’s inbox. It is forty lines of quoted reply thread, a signature block with the rep’s title, phone, and a legal footer, some broken HTML from the customer’s mail client, the words “see attached,” and a screenshot of a checkout error. The customer’s actual problem — three failed payment attempts and an error code — is two sentences buried in the middle, and the error code exists only in the image. Pass that raw blob to the Module 2 triage and the classifier drowns in the signature: it returns intent other, priority low, and never sees the error at all. The best prompt in the world does not repair a polluted input. Garbage in, garbage out is not a slogan here; it is a billed API call that misroutes a frustrated customer. This module builds the intake pipeline that validates, normalizes, and reads that screenshot before the foundation model (FM) ever sees the ticket.

In this module

You’ll learn how to:

Create a validation workflow that guarantees input quality before any foundation-model call, with explicit, structured rejections (skill 1.3.1).
Process complex data types — text plus image (an error screenshot) in practice, audio as a survey — through a multimodal Bedrock model (skill 1.3.2).
Format input to a model’s requirements: Converse content blocks, image constraints, multi-turn conversation formatting (skill 1.3.3).
Enhance input quality with normalization and Amazon Comprehend entity extraction (skill 1.3.4).
Evaluate where Bedrock Data Automation fits a document-processing system (skill 2.5.3) — and where Amazon Textract, the in-scope OCR service it sits beside, does not.

You’ll build: Relay’s intake pipeline — raw email/chat plus a screenshot becomes a validated, normalized Ticket, with clean rejections for bad input.

Exam domains covered: D1 — Foundation Model Integration, Data Management, and Compliance (31% of the exam), the full Task 1.3. This module also reaches into Domain 2 (26%) for one skill, 2.5.3 (document processing and Bedrock Data Automation).

Prerequisites: Modules 1–5. An AWS_PROFILE that resolves to your course account, and us-east-1 everywhere. You need the Module 2 Ticket/Triage schemas, the Module 3 converse() layer, and the Module 4 data bucket.

Where you are in the build

This is one stop on a 16-module road. M1–M5 are behind you; M6 is here; M7–M16 are ahead.

✅ M1–M5 — secured account, structured triage, the converse() model router, RAG ingestion, and a cited Knowledge Base.
👉 M6 — the intake pipeline. Validation, normalization, entity extraction, and a multimodal screenshot read, all upstream of triage.
⬜ M7–M16 — agents and tools, safety, security, deployment, cost, evaluation, observability, and the capstone.

Relay before this module: it triages and answers cited questions, but only on clean strings someone handed it. Relay after this module: relay/intake.py sits at the head of the chain, turning real-world email, chat, and screenshots into a validated Ticket the rest of the system can trust.

Validate before you generate: input quality gates

The first rule of a foundation-model pipeline is the cheapest one: a data validation workflow — a set of checks that guarantee input quality before consumption — runs before the FM, never after. The exam tests where you place it, and the answer is always “upstream,” for two reasons it keeps coming back to.

Cost. Every Converse call and every Comprehend call on garbage is billed. A customer pastes a 16 KiB runaway log and someone triages it “just in case” — at one ticket that is nothing, at a million tickets it is real money on inputs you would never have wanted to process. Validation is the only step in the pipeline that saves money by running.

Integrity. An invalid input that is not rejected does not disappear; it becomes a silent error somewhere downstream. An empty body produces an empty FM call; a binary blob mislabeled as text produces a hallucinated triage. The foundation model is not an input validator. The pipeline before it is.

So Relay’s intake gates the input and, on failure, raises a structured business error — not a bare ValueError, not a swallowed exception. The CLI prints it and exits non-zero. There is no silent try/except; an invalid input is rejected loudly, with a machine-readable reason code you can route on. The gates are deliberately boring:

Gate	Rejects	`reason` code
UTF-8 decode	a binary blob mislabeled as text	`bad_encoding`
size ceiling (16 KiB)	a runaway log or thread paste	`message_too_large`
non-empty after normalize	a body that is all signature/thread	`empty_message`
admitted image type	a PDF / zip / `.exe` attachment	`bad_attachment_type`
attachment size (4 MB)	an over-size image	`attachment_too_large`
non-empty attachment	a zero-byte attachment	`attachment_empty`
channel	anything but `email` / `chat`	`bad_channel`

There is a second exam distinction hiding in this section, and it trips people up. The blueprint mentions Glue Data Quality and SageMaker Data Wrangler under input validation — but those validate datasets. AWS Glue Data Quality runs rules over a table in a data pipeline; SageMaker Data Wrangler profiles and cleans features for a training job. Both are batch, both operate on a whole dataset, and neither belongs in a per-request path. Relay validates one ticket as it arrives, which is custom code — Lambda-style, synchronous, sub-millisecond. When the exam describes “a streaming support intake that must reject malformed messages,” the answer is custom validation in the request path, not a Glue job. Match the granularity: dataset versus request.

⚠️ Common misconception: “The FM will handle messy input — that’s what it’s for.” It will not. Noise costs tokens you pay for, dilutes attention (a 40-line signature genuinely buries a two-line problem in the model’s context), and an invalid input that is never rejected becomes a silent downstream error. A foundation model is an interpreter, not a validator: it has no concept of “this should have been rejected.” The pipeline in front of it is the validator — and the cheaper it runs, the less the FM ever sees garbage.

One ticket, many shapes: formatting input for the model

Once an input passes the gates, you have to format it the way the model expects. The Converse API speaks in content blocks — the typed units (text, image, document, tool result) that make up one message. A single message is {"role": ..., "content": [...]}, and that content list can hold more than one block at once. That is the whole trick to multimodal input on Bedrock: a user message carries a text block and an image block together, and the model reasons over both.

The other piece of formatting the exam loves is conversation formatting — the discipline of structuring a multi-turn exchange as an alternating sequence of user and assistant messages, in order, so the model reads the history correctly. Converse is strict about it: roles alternate, the list starts with user, and a tool result rides in the turn that follows its tool call. Relay’s intake is single-turn for now (one ticket in, one structured read out); the multi-turn, tool-driven conversation is the agent in Module 7. But the shape is the same shape, and getting it wrong — two user messages back to back, a mangled history — is a validation error the API returns, not a model quality problem.

Here is the intake pipeline as a whole, with the reject branch that makes it production-grade:

flowchart TD
    A[Raw email / chat + optional screenshot] --> B{Validate<br/>encoding · size · channel · attachment type}
    B -->|fails a gate| R[IntakeRejected<br/>structured reason · exit 1]
    B -->|passes| C[Normalize<br/>strip signature · quoted thread · HTML]
    C --> D{Non-empty?}
    D -->|empty| R
    D -->|content| E[Comprehend detect_entities<br/>order #, dates, products → Entities line]
    E --> F{Attachment?}
    F -->|none| H[Build validated Ticket]
    F -->|screenshot| G[Upload to attachments/ → Nova Lite vision read<br/>→ Attachment summary appended]
    G --> H
    H --> I[Module 2 triage]

The body gates — encoding, size, channel, non-empty after normalization — all run first, before any billed call. The attachment’s own cheap gates (admitted image type, size) run inside the attachment branch, just before the upload and the vision read, so a bad attachment is still rejected for free before Relay spends anything on it. (One honest caveat: in the shipped code the Comprehend entity call runs just before the attachment branch, so a ticket with a valid body but a rejected attachment does pay for that one Comprehend call first — a cheap, bounded cost the order keeps small.) The order is also the order Module 10’s PII redaction will need — it slots in right after normalization, before Comprehend and before the vision call — which is why intake is structured as discrete, ordered steps rather than one big function.

Multimodal input: making Relay read screenshots

This is the headline increment. CloudCart’s customers cannot describe an error they do not understand; they screenshot it. The error code, the screen, the instruction the page gives them — all of it lives in the image and nowhere in the text. Multimodal input, here, means the model reads pixels and prose in the same request.

Relay uses Amazon Nova Lite (the vision tier in relay/config.py, us.amazon.nova-lite-v1:0) for this. Note the name carefully: Nova Lite is not the smart tier’s Nova 2 Lite — they are different models, and the exam’s model-selection questions punish confusing them. The vision model ID, like every model ID in Relay, lives only in config.py; intake never writes a bare ID.

The image goes through the Converse content-block path, never a legacy single-prompt invoke_model with a base64 blob in a model-specific body. relay/llm.py owns the one function that knows the Converse image shape:

IMAGE_MEDIA_TYPE_TO_FORMAT = {
    "image/png": "png", "image/jpeg": "jpeg",
    "image/gif": "gif", "image/webp": "webp",
}

def image_block(data: bytes, media_type: str) -> dict[str, Any]:
    """One Converse IMAGE content block from raw bytes + MIME type."""
    fmt = IMAGE_MEDIA_TYPE_TO_FORMAT.get(media_type)
    if fmt is None:
        raise ValueError(f"Unsupported image media type {media_type!r}.")
    # Raw bytes: boto3 base64-encodes for the wire. We never build a
    # model-specific single-prompt invoke payload (banned).
    return {"image": {"format": fmt, "source": {"bytes": data}}}

Intake builds one message that carries a text instruction and this image block, then sends it through the single Bedrock call site, converse(tier="vision"). The prompt is narrow and grounded — “read what is visible,” not “imagine what might be wrong”:

def read_screenshot(data: bytes, media_type: str) -> str:
    """Read an error screenshot with Nova Lite (vision tier)."""
    message = {
        "role": "user",
        "content": [
            {"text": _VISION_PROMPT},          # "Error / Screen / User action"
            llm.image_block(data, media_type), # the image, same message
        ],
    }
    result = llm.converse(
        [message], tier=config.VISION_TIER,
        inferenceConfig={"maxTokens": 220, "temperature": 0.0},
    )
    return result.text.strip()

The short, structured extraction is appended to customer_message under an [Attachment summary] separator. Critically, the Ticket schema does not change to hold it: the summary rides inside the existing customer_message string, while the file’s metadata rides in the new Ticket.attachments list. The model gets to see the error; the schema stays frozen.

Image-format constraints (skill 1.3.3). Converse accepts png, jpeg, gif, and webp via an image block, and the intake gate admits exactly those four — one source of truth, shared between “what we accept” and “what we can send.” Image size and per-request count are model-side limits; re-verify the current Nova Lite figures on the Amazon Nova docs before you push large or many images. Relay reads one screenshot per ticket.

Audio is a survey, not a lab. Skill 1.3.2 names audio, so know the path: it is Amazon Transcribe → text → FM, not “send the audio straight to the model.” Transcribe converts speech to text, and that text becomes a normal text block. Module 6 builds the image path only; the audio path is the same idea with a transcription step in front.

⚠️ Common misconception: “Textract reads the screenshot.” It does not, and the exam sets this trap deliberately. Amazon Textract does OCR — it extracts text, tables, and form fields out of a document (a scanned PDF, an invoice). Point it at payment_error.png and it might return the literal string “ERR-402,” but it cannot tell you that ERR-402 means a declined payment on the checkout screen. Reading text is not understanding context. Interpreting a screenshot — what the error means, which screen it is, what the user should do — is the job of a multimodal FM, not an OCR engine.

Beyond DIY: Comprehend, Textract, and Bedrock Data Automation

The intake also runs entity extraction with Amazon Comprehend — detect_entities pulls order numbers, amounts, dates, and product names out of the normalized text, and intake appends them as a structured [Entities] line so triage and the agent get the salient facts up front. Comprehend is a managed NLP service, not a Bedrock foundation model and not a guardrail. It is the right tool when you want targeted, known-shape facts out of text without paying for FM tokens or prompt engineering.

So you have four overlapping options for “get structured information out of this input,” and the exam’s favorite question shape is matching the input to the service. Here is the matrix the lab’s Step 6 points at:

	Amazon Comprehend	Amazon Textract	FM multimodal (Nova Lite)	Bedrock Data Automation
What it does	Entity / sentiment / language detection in text	OCR: text, tables, forms from documents	Contextual understanding of image + text	Managed end-to-end document pipeline (parse → extract → transform)
Input type	Plain text	Scanned PDFs, images of documents	Images + text in one Converse message	Documents, images, audio, video at scale
When to choose	Targeted facts from a known-shape message	Get text/structure out of a document	Interpret what an image means	Industrialize a multi-step document workflow
Relative cost	Low (per 100-char unit)	Low–moderate (per page)	Low (per-token Converse)	Moderate (managed, per asset)
CloudCart example	Order #, dates from a ticket	OCR a scanned refund form	Read the checkout error screenshot	Process a batch of supplier invoices

The exam’s logic, in one line each: targeted entity extraction is Comprehend; OCR of structured documents is Textract; contextual understanding of an image is a multimodal FM; an end-to-end, managed document pipeline is Bedrock Data Automation — the service that parses, extracts, and transforms documents (and other media) into structured output without you wiring Textract, Comprehend, and an FM together by hand. Module 6 covers BDA as theory only; it runs no real BDA or Textract job (budget, and re-verify BDA status and pricing as of June 2026 before any costed claim). Orchestrating these steps into a workflow — Step Functions, EventBridge — is Module 11.

Build it: an intake pipeline for Relay

Goal: build relay/intake.py so that raw email/chat plus an optional screenshot becomes a validated, normalized Ticket — and so invalid input is rejected cleanly with exit code 1.

This lab cost me $0.01 (measured end to end on 13 June 2026, us-east-1, well under the Module 6 budget of < $1). The spend is a handful of Nova Lite vision reads, Comprehend entity calls, one Module 5 KB answer in the live smoke test, and a couple of S3 uploads. Every token figure below is read from the API response.

Step 1 — Carry the cumulative state forward. Module 6 starts from Module 5’s relay/ package byte-for-byte and inserts upstream of it. uv sync (no new runtime dependency), then aws sts get-caller-identity to confirm the account.

Step 2 — The frozen contract. relay/models.py gains the Attachment schema and one new field on Ticket, by addition only:

class Attachment(BaseModel):     # frozen M6 — exactly 3 fields
    filename: str
    media_type: str
    s3_uri: str

class Ticket(BaseModel):
    ticket_id: str
    channel: Literal["email", "chat"]
    customer_message: str
    attachments: list[Attachment] = []   # ADDED M6 (default [] — back-compat)
    created_at: str

The default [] is load-bearing: every Module 2–5 ticket fixture (none has an attachments key) still validates. There is no pii_redacted field — that is a Module 10 addition. The intake normalizes the message but redacts nothing.

Step 3 — Validate before you generate. The gates from the table above, each raising a typed IntakeRejected(reason=...). The two rejections, live:

uv run python -m relay.intake data/raw/invalid_oversized.txt
# REJECTED (message_too_large): Message is 43889 bytes, over the 16384-byte limit...
echo $?   # 1

uv run python -m relay.intake data/raw/invalid_empty.txt
# REJECTED (empty_message): Message is empty after normalization...
echo $?   # 1

Step 4 — Normalize, then extract entities. normalize() strips the signature, the quoted reply thread, and simple HTML, then collapses whitespace — fewer tokens, more signal. Then Comprehend appends an [Entities] line. On the real billing fixture, detect_entities returned:

quantity: three times, $49.00; date: today, 2026-06-12; organization: CloudCart; other: #1042

Step 5 — Read the screenshot. The full end-to-end command, with the attachment and a triage pass, against the real account:

uv run python -m relay.intake data/raw/email_billing_error.txt \
    --attachment data/raw/payment_error.png --triage

The PNG uploads to s3://relay-<account_id>/attachments/<uuid>-payment_error.png (3,816 bytes, verified present then purged at teardown). Nova Lite reads it — 808 input + 38 output tokens, a small fraction of a cent — and the [Attachment summary] block appears in customer_message. The error code lived only in the image:

[Attachment summary]
Error: ERR-402 Payment declined
Screen: CloudCart checkout
User action: use a different card and click Retry payment

The exact wording varies run to run — Nova Lite is non-deterministic at the margins — but the load-bearing fact (the code ERR-402 lived only in the image) is stable, which is what the test asserts.

Then triage runs on the clean ticket and returns the right routing — {"intent": "billing", "priority": "high", "sentiment": "negative"} (~1,360 in / ~22 out tokens). All three valid fixtures triaged to a real intent (billing / shipping / account) at high priority — none fell into the other/low hole the raw email did. That contrast is the module: the fix is the pipeline, not a bigger prompt.

Step 6 — Offline tests, then teardown.

uv run pytest                              # offline, no creds, no network (Modules 2–6)
RELAY_LIVE_TESTS=1 uv run pytest -m live    # up to 6 sub-cent real calls
uv run python teardown.py                  # purge attachments/ uploads; KEEP bucket + KB

Offline runs 107 passed, 5 skipped (the live tests, correctly skipped). The live marker runs 5 passed, 107 deselected in ~6.3s — including the Module 6 Nova Lite vision read asserting it surfaced ERR-402 / declined / payment. Teardown purges the attachments/ uploads and keeps the data bucket, the docs/ corpus, and the Knowledge Base (downstream modules reuse them, and they idle at ~$0). Comprehend and Converse are per-call, so nothing else needs deleting.

Try it yourself. (a) Add a language gate with Comprehend detect_dominant_language and reject a non-English ticket with a clear IntakeRejected — does it belong before or after detect_entities? (b) Feed a busier screenshot with several error dialogs at once and watch where the structured extraction degrades. A clean, single-error screenshot is the reliable case; this is the precision boundary of multimodal input.

In production

At CloudCart scale, intake is not a synchronous CLI call. Tickets land on a queue and the pipeline runs asynchronously, so a slow vision read never blocks the API (the queue and the API are Module 11). Rejections are not just logged and dropped — you quarantine them. A rejected input goes to a dead-letter store with its reason code, so support can audit what got bounced and tune the gates; a spike in message_too_large might mean a broken client, not a bad customer.

Cost discipline matters more here than the lab suggests. Comprehend bills per unit of 100 characters with a three-unit (300-character) minimum per request, roughly $0.0001 per unit as of June 2026 — re-verify on the Comprehend pricing page. In the lab, Comprehend was actually the single biggest line, ahead of the vision reads, because of that minimum. At a million tickets a day that minimum is the number you optimize, perhaps by batching short messages or skipping Comprehend on tickets below a length threshold.

Two things stay firmly out of scope and you should say so out loud. First, a malicious attachment is a security problem — virus scanning, content-type spoofing — handled by an antivirus scan on upload, not by anything GenAI. Second, this pipeline does not redact PII; it normalizes, but a normalized email still contains the customer’s name and address. In Module 10, PII redaction slots into this exact pipeline, on the normalized text, before any FM call.

Exam corner

What the exam tests here. Task 1.3 in full: where validation lives and how to reject (1.3.1), how to process complex types like image and audio (1.3.2), how to format input to a model’s requirements including Converse content blocks and conversation formatting (1.3.3), and how to enhance input quality with normalization and Comprehend (1.3.4). Plus the document-processing slice of skill 2.5.3 — Bedrock Data Automation; Amazon Textract is the in-scope OCR service it sits beside in the matching questions. Expect at least one matching question (service ↔ step).

Quiz.

A streaming support intake occasionally crashes a downstream Lambda when a customer pastes a 50 KiB log. The team wants to stop processing these before any model call. What is the correct fix?
- A. Increase the Lambda memory and timeout so it can process the large input.
- B. Add a size gate in the request path that rejects oversized input with a structured error, before the FM call.
- C. Wrap the FM call in a try/except that returns a default triage on failure.
- D. Run a nightly Glue Data Quality job over the ticket table.
A ticket arrives with a scanned PDF refund form, a free-text note that needs order numbers pulled out, and a screenshot of an error dialog. Match each to its best AWS service.
- A. Textract for all three.
- B. PDF → Textract (OCR); note → Comprehend (entities); screenshot → multimodal FM (interpretation).
- C. PDF → Comprehend; note → Textract; screenshot → Textract.
- D. Bedrock Data Automation for all three, individually, per request.
A developer’s multimodal Converse call returns a validation error before reaching the model. They are sending a .bmp screenshot, raw, as the only content block. What is the most likely cause?
- A. The image must be a URL, not raw bytes.
- B. The image format is unsupported; Converse images must be png, jpeg, gif, or webp.
- C. Converse cannot carry an image and text in the same message.
- D. The model needs the image base64-encoded by the caller before sending.
A chatbot’s multi-turn history is failing Converse validation intermittently. Logs show two user messages sometimes appear in a row with no assistant message between them. What is wrong?
- A. The temperature is set too high for multi-turn input.
- B. The conversation formatting is invalid; Converse requires alternating user/assistant roles in order.
- C. The history exceeds the context window.
- D. Multi-turn conversations are unsupported by the Converse API.
Triage quality dropped after a customer base started sending long emails with signatures and forwarded threads. The team is tempted to write a longer, more defensive triage prompt. What is the better fix?
- A. A longer prompt that instructs the model to ignore signatures.
- B. A higher-tier, more expensive model for triage.
- C. A normalization + entity-extraction step upstream that strips the noise before the FM sees it.
- D. Lower the temperature on the triage call.

Answers. 1 — B. Validation belongs in the request path, before the FM, as a structured rejection; A wastes money, C hides errors, D validates the wrong granularity (a dataset, not a request). 2 — B. OCR a document with Textract, pull entities with Comprehend, interpret an image with a multimodal FM — the canonical three-way match. 3 — B. Converse admits png/jpeg/gif/webp; raw bytes are correct (the SDK base64-encodes for you, so D is wrong), and an image can share a message with text (C is wrong). 4 — B. Conversation formatting requires alternating roles; two user turns in a row is malformed history, not a context or temperature issue. 5 — C. Normalize and extract entities upstream; a bigger prompt or a bigger model pays to compensate for noise you should have stripped for free.

Traps to avoid.

Textract is not image understanding. Textract extracts text and structure from documents; interpreting a screenshot (“this means a declined payment”) needs a multimodal FM. The exam pairs these on purpose.
Audio does not go straight to the model. The exam path is Transcribe → text → FM. Treat “send the audio to the model” as wrong unless a question explicitly establishes native audio support.
Validation is not security. Integrity validation (Module 6) and guardrails for prompt injection / unsafe content (Domain 3, Module 9) are two distinct layers. A well-formed input can still be an attack; a malformed one is rejected before either runs.

Key takeaways

Validate before you generate. Every gate runs before the FM, because every call on garbage is billed and every unrejected invalid input is a silent downstream error.
Reject explicitly. A structured business error with a machine-readable reason and exit 1 — never a swallowed exception, never a coerced default.
Normalization is a token lever. Stripping signatures, quoted threads, and HTML means fewer tokens and more signal; it fixes triage that a bigger prompt cannot.
One Converse message carries text and an image. Multimodal input is just two content blocks in one message; the image is raw bytes through image_block, never a legacy invoke payload.
Match the service to the step. Comprehend extracts entities, Textract OCRs documents, a multimodal FM understands an image, and Bedrock Data Automation industrializes the whole document pipeline.
Audio goes through Transcribe first. Speech becomes text, then text goes to the FM — there is no “audio straight to the model” path here.
Intake is the slot for what comes next. Module 10’s PII redaction drops in after normalization, before any FM call — the pipeline is built ordered so it fits.

What’s next

Relay can now ingest anything CloudCart throws at it — messy email, broken HTML, an error screenshot — and turn it into a clean Ticket. But it still only answers. Module 7 turns Relay into an agent that acts: looking up orders, creating tickets, and calling tools over MCP with Strands Agents.

Want the full AIP-C01 question bank and the next module in your inbox? Subscribe here — it’s free, like everything in this series.

Links: Module 6 lab · ← Module 5 · Course index · Module 7 →

References

Amazon Bedrock Converse API — image and document support — content blocks, multimodal messages, and conversation formatting.
Amazon Nova user guide — Nova Lite vision capabilities and current image size/format limits.
Amazon Comprehend developer guide — detect_entities, entity types, and the per-unit billing model.
Bedrock Data Automation — managed end-to-end document and media processing; re-verify status and pricing.
AWS Certified Generative AI Developer - Professional (AIP-C01) exam guide — Task 1.3 and skill 2.5.3, the vocabulary this module covers.
Amazon Bedrock samples — reference implementations for Converse, multimodal input, and document processing.