Immediacy is assumed, not required
Features get built to respond instantly because that is how software usually behaves. But a great deal of AI work — categorising, summarising, enriching, scoring — is consumed hours later by a person who never noticed when it ran.
Batch is cheaper and calmer
Running work in bulk lets you control concurrency, retry failures without a user waiting, and stay inside provider limits by design rather than by luck — the load problems that break real-time features mostly disappear.
If nobody is watching the spinner, you don’t need a spinner.
The test is who is waiting
If a person is sitting in front of it, latency matters and you design for the wait. If the output lands in a queue, a report or a record, then finishing by nine tomorrow is indistinguishable from finishing now.
Batch makes review practical
Work produced overnight can be sampled and corrected before anyone acts on it, which converts a risky autonomous step into a draft-and-approve one without changing the model at all.
It needs its own monitoring
The cost of moving work to a schedule is that failures become silent. Batch jobs need alerting on absence and completion, not just on errors — the scheduled-job discipline applies fully.
The hybrid is common
Pre-compute what you can overnight and handle only the genuinely novel request live. That keeps interactive latency low, cost predictable, and the expensive path narrow.