~/writing/math-foundations-for-reasoning-models $ ls
Structured Neural Architectures
Pooling
Pooling summarizes nearby feature values to reduce detail while keeping useful local signals.
~/writing/math-foundations-for-reasoning-models $ cat topic.md
Pooling
Definition
Pooling is an operation that summarizes nearby values.
Basic idea:
local group of values -> summary value
Example summaries:
maximum value
average value
Pooling reduces local detail into a smaller representation.
Why Pooling Is Used
Pooling can reduce the size of a representation.
Example:
large feature map -> smaller feature map
This can reduce computation and make later layers work with more compact information.
Pooling compresses local feature information.
Max Pooling
Max pooling takes the largest value in a local region.
Example:
[0.1, 0.7, 0.2, 0.4] -> 0.7
This keeps the strongest local response.
Max pooling asks which feature response was strongest nearby.
Average Pooling
Average pooling takes the average of nearby values.
Example:
[0.1, 0.7, 0.2, 0.4] -> 0.35
This preserves a smoother summary of the region.
Average pooling summarizes the overall local response.
Final Summary
- Pooling summarizes nearby feature values.
- It can reduce representation size.
- Max pooling keeps the strongest local response.
- Average pooling keeps an average local response.
- Pooling removes some detail while preserving useful signals.
- It is often used with convolutional representations.
~/writing/math-foundations-for-reasoning-models $ open
Continue the series.
Translation Invariance