~/writing/math-foundations-for-reasoning-models $ ls
Embeddings and Representation
Similarity in Embedding Space
Similarity in embedding space measures how close or aligned learned representations are inside the model.
~/writing/math-foundations-for-reasoning-models $ cat topic.md
Similarity In Embedding Space
Definition
Similarity in embedding space measures how related two representations are geometrically.
Common ideas:
close together
pointing in a similar direction
high alignment
The exact measurement depends on the method used.
Similarity is a geometric relationship between embeddings.
Why Similarity Matters
If embeddings are numerical representations, the model can compare them.
Example:
embedding A compared with embedding B
This comparison can support:
retrieval
classification
attention
clustering
prediction
Similarity helps the model reuse structure across related representations.
Similarity Does Not Mean Identical
Two embeddings can be similar without being the same.
Example:
similar direction
different exact coordinates
This matters because related items can share patterns while still preserving differences.
Similarity means related representation, not duplicate representation.
Similarity Is Model-Relative
Embedding similarity depends on the model and training data.
Two representations may be similar because they are useful in similar computational contexts.
That does not automatically mean they are similar in every human sense.
Embedding similarity reflects learned representational structure.
Example
Suppose two tokens often appear in similar contexts.
Training may push their embeddings into a related region.
Simple idea:
similar use -> similar representation
This is not guaranteed, but it is a common useful pattern.
Similarity often reflects shared usage patterns.
Final Summary
- Similarity is a geometric relationship between embeddings.
- Similar representations may be close or aligned.
- Similar does not mean identical.
- Similarity is model-relative.
- It can help models compare, retrieve, attend, and generalize.
~/writing/math-foundations-for-reasoning-models $ open
Continue the series.
Analogy in Vector Space