~/writing/math-foundations-for-reasoning-models $ ls
Information Theory and Loss
Surprise
Surprise is the intuition behind self-information: unlikely outcomes are more surprising when they happen.
~/writing/math-foundations-for-reasoning-models $ cat topic.md
Surprise
Definition
Surprise is the intuitive name for how unexpected an observed outcome is.
In information theory:
surprise = self-information = -log(probability)
Surprise measures how unexpected an outcome was under a probability model.
Likely Means Low Surprise
If a model assigns high probability to an outcome, observing that outcome is not very surprising.
Example:
P(rain | dark clouds) = 0.9
If it rains, the surprise is low.
Expected outcomes carry low surprise.
Unlikely Means High Surprise
If a model assigns low probability to an outcome, observing that outcome is surprising.
Example:
P(rain | clear sky) = 0.01
If it rains, the surprise is high.
Unexpected outcomes carry high surprise.
Surprise Depends On The Model
The same event can be surprising under one model and unsurprising under another.
Example:
Model A: P(event) = 0.9 -> low surprise
Model B: P(event) = 0.01 -> high surprise
The event did not change.
The probability assigned to it changed.
Surprise is relative to the probability model.
Surprise And Prediction
When a model predicts a distribution, the true outcome should ideally receive high probability.
Example:
true label = bug
model assigns P(bug) = 0.8
Surprise is low.
If:
model assigns P(bug) = 0.01
surprise is high.
Good predictions make the true outcome less surprising.
Surprise As Error Signal
Surprise can become an error signal.
If the correct answer is highly surprising under the model, the model's probability assignment is poor.
That poor assignment can be turned into loss.
high surprise -> high loss
low surprise -> low loss
Loss can penalize a model for being surprised by the truth.
Surprise vs Confusion
Surprise here is not a feeling.
It is a mathematical quantity based on probability.
The model does not need awareness.
It only assigns probabilities, and the observed outcome is evaluated against them.
Mathematical surprise does not require conscious surprise.
Final Summary
- Surprise measures unexpectedness under a probability model.
- It is equivalent to self-information.
- Likely outcomes have low surprise.
- Unlikely outcomes have high surprise.
- Surprise depends on the model's probability assignment.
- High surprise for the true outcome can become high loss.
~/writing/math-foundations-for-reasoning-models $ open
Continue the series.
Entropy