Validation Data

Definition

Validation data is held-out data used during model development.

Simple form:

train model on training data
check choices on validation data
save test data for final evaluation

Validation data helps choose model settings.

Validation data guides development without using the final test set.

---

Why Validation Data Is Needed

Model building involves choices.

Examples:

model size
learning rate
regularization strength
number of training steps
architecture choice

Validation data gives feedback on these choices.

Validation data helps choose among possible models.

---

Training vs Validation vs Test

The three sets have different roles.

training data:
updates parameters

validation data:
guides model selection

test data:
estimates final generalization

Keeping the roles separate makes evaluation more trustworthy.

Data splits protect against fooling yourself with reused evidence.

---

Validation Can Also Be Overused

If many decisions are made using the same validation set, the development process can overfit to it.

Example:

try many models
keep the one with best validation score

The chosen model may fit the validation set better than it fits future data.

This is why a final test set is still needed.

Validation data guides decisions, but it is not a perfect guarantee.

---

Example

Suppose a model is trained for many epochs.

Training loss keeps going down.

Validation loss starts increasing.

This can signal overfitting.

training improves
validation worsens

That feedback can guide early stopping.

Validation data helps detect when training stops improving generalization.

---

Final Summary

• Validation data is used during development.
• It helps choose model settings and training decisions.
• It is separate from training data and final test data.
• It can still be overused.
• It helps estimate generalization while building the model.