Backward Pass

Definition

The backward pass computes gradients by moving backward from the loss through the network.

Forward pass:

input -> layers -> output -> loss

Backward pass:

loss -> output layer -> hidden layers -> earlier parameters

The backward pass traces how loss depends on earlier computations.

---

Why It Goes Backward

The loss is computed at the end of the forward pass.

But the parameters that affected the loss are spread throughout the network.

So the backward pass starts from the loss and works back through the chain of computations.

loss depends on output
output depends on hidden values
hidden values depend on weights and biases

Backward means following dependency from loss back to parameters.

---

What It Computes

The backward pass computes gradients.

For each trainable parameter, it asks:

if this parameter changes slightly, how does loss change?

The answer is a gradient.

parameter -> gradient

The backward pass produces the information needed for parameter updates.

---

Backward Pass Uses Forward Values

The backward pass often needs values produced during the forward pass.

Examples:

layer outputs
activation values
intermediate scores

These values help compute how later changes connect to earlier parameters.

The backward pass depends on the computation path created by the forward pass.

---

Backward Pass vs Parameter Update

The backward pass computes gradients.

The optimizer uses those gradients to update parameters.

backward pass:
compute gradients

update step:
change parameters

They are connected, but they are not the same operation.

Backward pass calculates; the optimizer updates.

---

Final Summary

• The backward pass starts from loss and moves backward through the network.
• It follows dependency from output back to parameters.
• It computes gradients for trainable parameters.
• It uses values from the forward pass.
• The backward pass does not update parameters by itself.
• Parameter updates happen after gradients are computed.