Model Output As Distribution

Definition

A model output can be a probability distribution over possible answers.

Instead of returning only:

class A

the model may return:

class A: 0.7
class B: 0.2
class C: 0.1

A distributional output represents uncertainty across possible answers.

---

Hard Output vs Distribution

Hard output:

bug report

Distributional output:

bug report: 0.7
feature request: 0.2
question: 0.1

The hard output gives one choice.

The distribution shows confidence and alternatives.

A distribution gives more information than the final selected label.

---

Classification Example

Suppose a model classifies a message.

Possible labels:

bug
billing
feature
question

Output:

bug: 0.6
billing: 0.1
feature: 0.2
question: 0.1

The model chooses bug if taking the highest probability.

But it also shows uncertainty.

Classification can be viewed as a distribution over labels.

---

Why This Matters

Uncertainty matters when decisions have different costs.

Example:

bug: 0.45
billing: 0.40
feature: 0.10
question: 0.05

The top answer is bug, but billing is close.

A system may ask for more information instead of acting confidently.

Distributional output supports better decisions under uncertainty.

---

Probability And Loss

If the correct label is bug, a model should assign high probability to bug.

If it assigns low probability to the correct label, the loss should be high.

Basic idea:

better distribution -> lower loss
worse distribution -> higher loss

Loss can evaluate the quality of a predicted distribution.

---

Output Distribution Is Not Understanding

A distributional output does not mean the model understands the answer.

It means the model produced a structured uncertainty estimate.

Example:

label probabilities

are still computed outputs.

Probability output is structured prediction, not awareness.

---

Bridge To Earlier Concepts

This connects several earlier ideas:

function -> maps input to output
probability -> output can express uncertainty
loss -> measures quality of the output
gradient -> can later adjust parameters

The model is still a function.

But its output can now be a distribution.

Probability expands what model output can represent.

---

Final Summary

• A model can output a probability distribution.
• A distribution shows uncertainty across possible answers.
• Hard decisions hide alternatives and confidence.
• Classification can be treated as a distribution over labels.
• Loss can evaluate predicted distributions.
• Distributional output does not imply understanding.