Training Loop

The iterative process where a model learns from data.

1. Forward Pass

Input data ($X$) is passed through the model to get a prediction ($\hat{y}$).

prediction = model(data)

The data flows through every layer. Matrix multiplications and activation functions are applied.

2. Loss Calculation (Loss Function)

Measures how wrong the model was by comparing the prediction ($\hat{y}$) to the actual target ($y$) using a Loss Function.

loss = loss_function(prediction, target)

Results in a single scalar number (e.g., 0.45). If this number is high, the model is wrong; if low, it’s right.

3. Zero the Gradients

Clear out the gradients calculated in the previous batch.

optimizer.zero_grad()

4. Backward Pass (Backpropogation)

Calculates the gradient of the loss with respect to every weight in the network.

loss.backward()

5. Weight Update / Optimizer Step

The optimizer updates the weights based on the gradients calculated in step 4.

optimizer.step()