Maximum Likelihood Estimation

Definition

Maximum likelihood estimation chooses parameters that make the observed data most likely.

Short form:

MLE

Basic goal:

choose parameters that maximize P(data | parameters)

MLE = choose the explanation under which the observed data is most probable.

---

Parameters As Hypotheses

Different parameter values imply different predictions.

Example:

parameter setting A -> data is likely
parameter setting B -> data is unlikely

MLE chooses the parameter setting that gives the highest likelihood to the data.

Parameters are judged by how well they explain observations.

---

Simple Coin Example

Suppose we flip a coin 10 times and observe:

7 heads
3 tails

Possible parameter:

p = probability of heads

MLE would choose a value near:

p = 0.7

because that makes the observed data most likely.

MLE fits parameters to observed frequencies.

---

Likelihood Function

The likelihood function treats the data as fixed and parameters as adjustable.

L(parameters) = P(data | parameters)

The question is:

which parameters make L largest?

A likelihood function scores parameter settings by data fit.

---

Why Log Likelihood Is Often Used

Likelihoods can involve multiplying many probabilities.

Multiplying many small numbers can become hard to work with.

Log likelihood turns products into sums.

maximize likelihood
same parameter choice as
maximize log likelihood

Log likelihood is usually easier to optimize.

---

MLE And Loss

Learning often minimizes loss instead of directly maximizing likelihood.

A common connection:

minimize negative log likelihood

Since maximizing log likelihood is equivalent to minimizing its negative.

Negative log likelihood turns likelihood maximization into loss minimization.

---

Why MLE Matters For Learning

Many learning methods can be understood as MLE.

Basic idea:

model assigns probabilities
data is observed
choose parameters that assign high probability to observed data

This connects probability to optimization.

MLE turns learning into probability-based parameter fitting.

---

Final Summary

• Maximum likelihood estimation chooses parameters that make observed data most likely.
• It maximizes P(data | parameters).
• Parameters are treated as possible explanations.
• Log likelihood is often used because it is easier to optimize.
• Minimizing negative log likelihood is equivalent to maximizing log likelihood.
• MLE connects probability, loss, and optimization.