Throughput
How much work a system handles per unit of time — tokens per second, requests per minute.
Throughput is the capacity question, distinct from latency’s speed question. Batching raises throughput while raising individual latency, which is exactly the trade you want for offline jobs and exactly wrong for chat.
In practice: Batch a million classifications overnight; nobody is watching the clock.