The set of training examples processed together in one forward and one backward pass, producing one weight update. The number of examples in it is the batch size.
Card 1 of 259. Answer: The set of training examples processed together in one forward and one backward pass, producing one weight update. The number of examples in it is the batch size.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
A batch is the group of training examples a network processes together in one forward and one backward pass, producing a single weight update. The number of examples in it is the batch size, a hyperparameter often confused with the batch itself.
Example: 32 RGB images of 224 × 224 pixels form a tensor of shape [32, 3, 224, 224]. The network makes 32 predictions at once, the loss is averaged over them, and backpropagation returns one gradient for that average. With 1,200 training images and a batch size of 32, an epoch has 37 full batches and a final batch of 16, so 38 updates, unless the partial batch is dropped (drop_last=True in PyTorch).
Terminology trap: in batch gradient descent the batch is the whole dataset, while everyday usage means a mini-batch. Defaults differ: Keras's fit uses 32, PyTorch's DataLoader 1. Activation memory grows roughly linearly with batch size, which usually caps it on a GPU. At inference, batching requests raises throughput but adds latency.
