Gradient

Definition

A gradient is a vector of partial derivatives.

For a function:

f(x, y)

the gradient contains:

change with respect to x
change with respect to y

Basic form:

gradient = [partial f / partial x, partial f / partial y]

Gradient = all local one-variable sensitivities collected into a vector.

---

Why The Gradient Is A Vector

A multi-input function can change in many directions.

Example:

x direction
y direction
combined x and y direction

The gradient uses a vector because direction matters.

It does not just say:

how much change exists

It also says:

which direction increases the function fastest locally

Gradients connect calculus back to vectors.

---

Simple Example

Function:

f(x, y) = x + 2y

Partial derivatives:

with respect to x: 1
with respect to y: 2

Gradient:

[1, 2]

The function increases more strongly in the y direction than the x direction.

The gradient encodes directional sensitivity.

---

Gradient At A Point

For some functions, the gradient changes depending on location.

Example:

f(x, y) = x^2 + y^2

Gradient:

[2x, 2y]

At (1, 1):

[2, 2]

At (3, 4):

[6, 8]

A gradient is local; it depends on the current point.

---

Gradient And Parameters

If a model has parameters:

w1, w2, b

and a loss depends on them:

loss(w1, w2, b)

then the gradient can be:

[change with respect to w1,
 change with respect to w2,
 change with respect to b]

This tells how the loss responds locally to each parameter.

A gradient is a compact description of parameter sensitivity.

---

Gradient Is Not The Update Yet

The gradient tells local direction and sensitivity.

It does not by itself decide:

how large a step to take
when to stop
which optimization method to use

Those questions belong to optimization.

Gradient is information; optimization uses that information.

---

Final Summary

• A gradient is a vector of partial derivatives.
• It describes local sensitivity in multi-input functions.
• It points in the direction of steepest local increase.
• It depends on the current point.
• For models, gradients describe how loss responds to parameters.
• Optimization later uses gradients to update parameters.