Standard Deviation

Standard deviation is the square root of variance, giving spread in the original units.

~/writing/math-foundations-for-reasoning-models $ cat topic.md

Definition

Standard deviation measures spread around the mean in the original units of the data.

It is the square root of variance.

standard deviation = sqrt(variance)

Standard deviation = typical distance from the mean.

Variance uses squared distances.

If the data is measured in:

meters

variance is measured in:

meters squared

That can be harder to interpret.

Standard deviation converts spread back into the original unit.

Standard deviation is often easier to interpret than variance.

Suppose variance is:

Then standard deviation is:

sqrt(9) = 3

This means values typically vary around the mean by about 3 units.

Standard deviation gives spread in familiar units.

Low standard deviation means values are close to the mean.

Example:

99, 100, 101

The values are tightly grouped.

Low standard deviation means low spread.

High standard deviation means values are more spread out.

Example:

50, 100, 150

The values are farther from the mean.

High standard deviation means high spread.

Variance:

average squared distance from mean

Standard deviation:

square root of variance

Both describe spread.

Standard deviation is usually easier to explain because it uses the original units.

Variance is mathematically convenient; standard deviation is more interpretable.

Standard deviation helps describe how much values fluctuate.

Examples:

feature values vary widely
loss values are unstable
predictions are tightly clustered

It is also used in normalization, where values are often centered and scaled.

Standard deviation helps compare spread across data and model behavior.

~/writing/math-foundations-for-reasoning-models $ open

Covariance