Foundations advanced

Batch Size

How many examples the model processes before updating its weights once.

Batch size trades stability against memory and speed. Large batches give smoother gradient estimates but need more GPU memory; small batches are noisier but sometimes generalise better. It interacts with learning rate, so the two are usually tuned together.

In practice: Batch size 32 means 32 examples in, one weight update out.

Where this comes up