It is 8:40 on a Sunday morning at a multi-branch dental group in Dubai. Forty-one patients have sent insurance cards overnight, most of them photographed at an angle in bad light, a few as PDFs pulled out of an email chain, two as 40-second voice notes explaining that the card is with a spouse. The automation is supposed to pull each attachment, file it against the patient record, and push a confirmation. Instead the ops lead is looking at a queue of eighteen records that say media_download_failed, and every one of them is now unrecoverable because the download link died while the job waited its turn. Nobody wrote bad code. They just treated media like text.
The short answer, stated up front
Handling WhatsApp Cloud API media messages at scale comes down to one architectural decision: when a media webhook arrives, download the file immediately into your own storage and never let your business logic touch Meta's servers again. Media URLs for incoming messages on the WhatsApp Cloud API expire after five minutes, while the media ID itself remains valid for 30 days, so the safe pattern is a fast, dumb fetch worker that resolves the ID to bytes within seconds, writes to object storage, and only then queues the interesting work (OCR, classification, routing, CRM writes). Every failure we are called in to fix traces back to a system that put processing before retrieval.
That is the whole answer. The rest of this article is why it works, the seven rules that surround it, and the places where reasonable engineers still disagree.
Commercial fishing crews have a discipline that matters more than any other on deck: the catch is worthless if it is not on ice before you sort it. You do not stand at the rail grading fish by size while the rest of the haul spoils in the sun. You get everything into the hold, cold, and then you grade at leisure. Media on WhatsApp works exactly the same way. The five-minute URL is your sun.
Expiry windows and why five minutes dictates your whole architecture
What actually expires, and when
Two clocks run on every inbound media message, and confusing them is the single most common design error we see. The media URL you retrieve from the Graph endpoint expires after five minutes. The media ID is good for 30 days. If you store the URL in your database and process the queue an hour later, you have stored a dead pointer. If you store the ID, you can re-resolve it to a fresh URL for the next month, which is a real safety net but not one to lean on for routine operation, because re-resolution costs an extra API call per file and those calls count against you when volume climbs.
The practical rule: persist the media ID, fetch the bytes now, store the bytes yourself. The ID is your receipt. The bytes are your inventory. Never confuse a receipt with inventory.
Why lazy loading fails here
Fetching on demand is a perfectly good pattern in most systems and a bad one on this platform. We have inherited builds where the agent inbox only fetched an attachment when a human clicked it. That works beautifully in testing, when the click comes ninety seconds after the message. It falls apart on Monday morning when a customer sent a document at 22:00 and the agent opens it at 09:00. The re-resolution path from the stored media ID saves you inside 30 days, so build it, but build it as a fallback rather than the primary route.
Size caps and MIME validation, the two checks that stop garbage entering your pipeline
The caps differ by media type
The Cloud API tops out at 100MB, but that ceiling is per document, not universal. Published guidance from SMS Gateway Center's media management breakdown puts images at 5MB, video at 16MB, and documents at 100MB, with audio formats including MP3, AAC, OGG and M4A. Those numbers matter in two directions. Outbound, a 6MB product image simply will not send, and if your CMS resizes images for the web but not for messaging you will discover this in production. Inbound, the caps work in your favour: WhatsApp compresses on the client side, so the images arriving in your webhook are considerably smaller than the originals on the customer's phone, which is why an insurance card photo often reads poorly under OCR.
Validate MIME type against content, not against the filename
The webhook payload tells you the MIME type. Trust it as a hint and verify it against the actual bytes before anything downstream opens the file. We have seen a PDF that was a renamed HEIC, a document that was a WhatsApp-forwarded sticker pack, and voice notes arriving as OGG when the parsing library was configured for M4A only. None of these are attacks. They are ordinary human behaviour on a consumer app, and your pipeline has to survive them without a dead-letter queue that nobody reads.
A short checklist we apply on every build:
- Reject and reply politely to anything over the type-specific cap, in the same conversation, within seconds. Silence is what makes customers resend the same file four times.
- Sniff the magic bytes; do not trust the extension or the declared MIME.
- Normalise voice notes to a single audio format at ingest so your transcription layer has one code path.
- Strip EXIF location data from customer photos before storage unless you have a documented reason to keep it.
- Cap total attachments per conversation per hour, because one confused customer can send sixty photos of the same document.
Retry logic that distinguishes a slow network from a dead link
Retries are where good intentions do the most damage. A blind exponential backoff with a base of thirty seconds will happily burn your entire five-minute window on two attempts and then fail permanently. The retry policy has to know which clock it is racing.
Inside the URL window, retry aggressively and tightly: three attempts at one, three and eight seconds, all against the same URL. If those fail, stop hammering. Go back to the media ID, request a fresh URL, and start a new short cycle. That second path is the one that saves you during Meta-side incidents, and those are real. A long-running Chatwoot issue thread documents media sending via the Cloud API failing intermittently for multiple users, with the community consensus favouring the upload-then-reference-media_id flow that n8n's WhatsApp node uses, precisely because it decouples file transfer from message send.
On the outbound side the same principle applies in reverse. Upload once, get a media ID, reuse that ID for every send within its 30-day life. A clinic sending the same pre-appointment PDF to 400 patients should upload it once, not 400 times. We have audited systems doing the latter, and the fix was an afternoon's work with an immediate drop in failed sends.
Storage offloading, or why your database should never hold a byte of media
The three-layer split
Media belongs in object storage. The message record in your database holds the media ID, your storage key, the MIME type, the byte size, a checksum, and the processing status. That is it. The moment someone base64-encodes a 14MB video into a Postgres column to keep things simple, every backup, every replication lag and every query plan in that system gets worse forever.
The three layers we install look like this: object storage for the raw file, a metadata row in the operational database, and a derived-artifacts table for whatever the AI produced (transcript, extracted fields, classification, confidence score). Keeping derived artifacts separate is what lets you re-run a better OCR model over eighteen months of insurance cards without re-downloading anything, because the raw bytes are still yours.
Retention, residency and the honest conversation nobody has early enough
Once you offload media, you own it, including the parts you would rather not own. Passport scans, medical images, trade licences and salary certificates all arrive on WhatsApp because customers find it easier than email. Decide the retention period before you go live, not after the first audit. For UAE clients we set explicit lifecycle rules on the storage bucket and document which region the objects sit in, because the answer to where a patient's X-ray physically lives is not something you want to be researching under pressure.
Webhook ordering, duplicates, and the assumption that quietly corrupts records
Webhooks are not ordered and they are not exactly-once. Meta's own documentation for the Cloud API message reference lists the full spread of message and status event types your endpoint has to tolerate, from image and video messages through to reactions, revokes and status updates. In practice this means a status webhook can arrive before the message webhook it refers to, and the same event can arrive twice.
Idempotency is the fix, and it is not optional at volume. Key every write on the WhatsApp message ID, use an upsert rather than an insert, and let late-arriving events update state rather than create rows. The webhook endpoint itself should do almost nothing: validate the signature, write to a queue, return 200 in single-digit milliseconds. WuSeller's guide to Cloud API throughput limits notes Meta targets median webhook latency under 250ms with fewer than 1% exceeding one second, and retries failed webhooks for seven days. That seven-day retry is generous and dangerous in equal measure: a slow endpoint that returns 200 late will be re-delivered the same events for a week, and without idempotency you will find duplicate patient documents multiplying quietly in the background.
The same guide points out the volume asymmetry that catches teams out: at 1,000 messages per second outbound you should size for roughly three times that in returning status webhooks, and one times for inbound replies. Media makes this worse, because each media message triggers your own fetch traffic on top of the webhook traffic.
Rate limits, throughput tiers, and pacing your media fetches
Two separate limits apply and teams routinely conflate them. Messaging tier governs how many unique customers you can message in 24 hours. Throughput governs messages per second. Fyno's developer guide on WhatsApp rate limits notes Cloud API throughput is commonly 80 messages per second by default for a business phone number, upgradeable to as much as 1,000 MPS when eligible. Media sends consume that same budget, and a broadcast of image templates behaves very differently from a broadcast of text.
Your inbound fetch workers need their own pacing. When a promotion lands and 900 customers reply with photos in ten minutes, an unbounded worker pool will open 900 concurrent downloads, saturate your egress, and start timing out inside the five-minute window. Bound the pool. Prioritise by age, oldest first, because the oldest file is the one closest to expiry. Trawlers do not shoot every net at once when the shoal appears; the winch has one speed, and overloading it costs you the whole haul.
If your media volumes swing hard with the calendar, and in the UAE they do around Ramadan, summer travel and the back-to-school weeks, the fetch pool should scale on the same rhythm as everything else. We wrote about designing for that in our piece on seasonal AI systems.
For teams already past the basics: encryption, resumable uploads and multi-number fleets
Running media across several business numbers
Once a group runs one number per branch, media handling gets a new failure mode: the same customer sends the same document to two numbers, and two separate records get created. Deduplicate on file checksum plus customer phone number, not on message ID, then link both conversations to one document. Routing rules matter here too, because a media message often needs a different owner than a text message. Our breakdown of inbox routing patterns covers the ownership logic that keeps one number sane across a team.
Large files and the upload path
For outbound documents approaching the 100MB ceiling, a single POST is a coin flip on a poor connection. Resumable upload sessions exist for exactly this, and they earn their implementation cost the first time a 60MB brochure dies most of the way through for the third time in a morning. For small files, the simple single-request path is fine and the added complexity buys nothing.
Where reasonable people disagree
Two arguments are genuinely unsettled among engineers who do this work. The first is whether to cache media ID to storage-key mappings across customers so an identical file sent by fifty people is stored once. It saves storage and it complicates deletion requests enormously; we usually decline it for regulated clients and accept it for retail. The second is whether to run AI processing synchronously with the fetch. The purist answer is no, always queue. The pragmatic answer is that a small clinic doing thirty documents a day gets a better customer experience from a two-second synchronous OCR than from a queue with an eight-second lag. We build synchronous below a defined threshold and switch to queued above it, and we have been argued out of that position more than once.
Unipile's overview of WhatsApp API integration makes the broader point well: business verification, template approval, webhooks, encryption and opt-in rules all have to be handled before a single file moves, and that upfront work is where most timelines slip.
The seven rules, condensed
- Fetch on arrival. The incoming media URL expires in five minutes; download to your own storage first and process second.
- Persist the media ID, not the URL. The ID stays valid for 30 days and is your only recovery path.
- Enforce type-specific caps. 5MB images, 16MB video, 100MB documents, checked before anything downstream opens the file.
- Verify MIME from the bytes. Declared types and filenames lie, routinely and innocently.
- Retry inside the window, then re-resolve. Tight retries against the URL, then a fresh URL from the ID, then stop.
- Keep media out of the database. Object storage for bytes, a metadata row for everything else, derived artifacts in their own table.
- Make every webhook write idempotent. Events arrive out of order, twice, and up to seven days late.
The part the architecture diagram will not tell you
Every rule above is engineering, and engineering is the easier half. The harder half is the reception desk. When a media pipeline goes live, the front-desk team stops receiving forty WhatsApp photos a day and starts receiving a clean list of processed documents, and the ones who understand why that happened will teach the next hire without being asked. The ones who suspect the system is auditioning for their job will find reasons it cannot be trusted, and they will be right often enough to be persuasive. We budget for that conversation deliberately, because change management decides more implementations than code does and it gets a fraction of the attention. Learnmind, the Dubai consultancy that builds WhatsApp automation and AI receptionists for service businesses, spends more hours on the training week than on the fetch worker.
The other discipline: automate the repetitive, personalise the meaningful. An insurance card arriving at 2am should be acknowledged, filed and validated by machine within seconds. A rejected claim, a scan that shows something serious, a document that fails validation for the third time, those get a human name attached. We have written elsewhere about the operational scaffolding that makes that split hold under load, in our notes on bulletproof client systems.
Common questions, answered
How long do WhatsApp Cloud API media URLs stay valid?
Media URLs for incoming WhatsApp Cloud API messages expire after five minutes, so the download must happen almost immediately after the webhook arrives. The associated media ID remains valid for 30 days, which lets you request a fresh URL if the first fetch failed.
What is the maximum file size for WhatsApp Cloud API media messages?
The overall ceiling is 100MB, which applies to documents, while images are capped at 5MB and video at 16MB. Validate against the type-specific limit rather than the 100MB figure, because most rejected sends are oversized images, not oversized PDFs.
Why do WhatsApp media sends fail intermittently even when the file is valid?
Intermittent media failures usually come from sending the file inline with the message instead of uploading it first and referencing the returned media ID. Decoupling upload from send, the flow documented in the Chatwoot issue thread, removes most of these failures and gives you a retryable ID.
If your WhatsApp volume is thirty documents a week and one person reads them all, do not hire us; a shared inbox and a folder will serve you better than anything we would build. If you are processing hundreds of images, PDFs and voice notes a day across multiple branches and losing some of them, that is a conversation worth having with our team.
