Batch Inference
Also called: offline inference
Running many predictions together as a job instead of one at a time in real time.
Batch inference processes a queue of inputs where latency does not matter, using hardware far more efficiently. Providers often discount it substantially. If a human is not waiting for the answer, it is the cheaper path.
In practice: Classifying a year of support tickets overnight at half price.