Yes. This is actually an important next step because it lets you explain why a recommendation system often prefers cosine similarity over a raw dot product.
You can add the following section after your dot-product example.
Dot Product vs. Cosine Similarity
The dot product gives us a useful compatibility score, but there is an important limitation.
The dot product depends on both:
- The direction of the vectors — whether the user and movie have similar preferences.
- The magnitude (length) of the vectors — how large the numbers are overall.
Cosine similarity, on the other hand, focuses primarily on the direction of the vectors.
The mathematical relationship
For two vectors and :
where:
- = dot product
- = magnitude of vector
- = magnitude of vector
- = angle between the two vectors
Cosine similarity removes the magnitude component:
Therefore:
For normalized vectors, the dot product and cosine similarity become the same value.
Why Does This Matter for Recommendations?
Let's make the difference very concrete.
Suppose we have two users.
User A
This user watches a lot of movies but strongly prefers Action and Sci-Fi:
User B
This user watches fewer movies but has almost exactly the same preference pattern:
Notice something interesting.
User B's vector is exactly half of User A:
So the two users have the same preference pattern, but User A has larger values.
Using Dot Product
Calculate:
The dot product is 73.
Now imagine another user:
Calculate:
So:
while:
The dot product strongly favors B.
But there is an important question:
Is B really more similar to A, or does B simply have large numerical values in the same dimensions?
Cosine Similarity Removes the Magnitude Problem
First calculate the magnitude of :
For B:
Now calculate cosine similarity:
That makes sense.
The two vectors point in exactly the same direction.
Even though their magnitudes are different, their preference patterns are identical.
An Even More Important Example
Now consider:
This could represent a user who has watched many more movies, but whose relative preferences are the same.
Notice:
Therefore A and D have exactly the same preference pattern.
Their cosine similarity is:
because they point in exactly the same direction.
But look at the dot product:
So we have:
| Comparison | Dot Product | Cosine Similarity |
|---|---|---|
| A vs B | 73 | 1.00 |
| A vs D | 1460 | 1.00 |
This is the key difference.
A and B are just as similar as A and D in terms of preference pattern, but the dot product gives D a dramatically larger score because D has a much larger magnitude.
Cosine similarity correctly recognizes that:
"The two users have the same preference direction, regardless of how large their individual values are."
Visual Intuition: Direction vs. Length
Think of each vector as an arrow.
The length of the arrow represents magnitude.
The direction of the arrow represents the pattern of preferences.
Two users could have:
User A: ───────────────► User B: ───────►
They have different magnitudes, but they point in the same direction.
Therefore:
and:
So their cosine similarity is:
This means maximum similarity.
Why Cosine Similarity Is Often Better for Recommendation
Suppose two users have the following preferences:
User 1
User 2
Their values are very different in magnitude.
But their preference pattern is identical.
Both users essentially say:
"I strongly prefer Action and Sci-Fi, have very little interest in Comedy and Romance."
Cosine similarity identifies this immediately.
The dot product, however, is influenced by the absolute magnitude of the vectors.
This can be problematic when the magnitude represents something unrelated to the actual similarity we want to measure.
For example, magnitude might be affected by:
- number of movies watched
- number of ratings given
- amount of interaction
- frequency of activity
- length of a document
- number of words in a document
In these situations, we often care more about the pattern of preferences than the absolute amount of activity.
That is where cosine similarity becomes particularly useful.
But Is Cosine Similarity Always Better?
No.
This is an important point for a machine-learning explanation.
Cosine similarity isn't universally better than dot product.
They answer slightly different questions.
Dot Product asks:
"How strong is the overall interaction between these two vectors?"
It considers both:
Cosine Similarity asks:
"How similar is the direction or pattern of these two vectors?"
It primarily considers:
So the choice depends on what the numbers represent.
A Simple Real-World Analogy
Imagine two customers buying products.
Customer A
Buys:
- 10 Action movies
- 2 Comedy movies
- 8 Sci-Fi movies
Customer B
Buys:
- 1 Action movie
- 0.2 Comedy movies
- 0.8 Sci-Fi movies
Customer B has the same preference pattern, but much less activity.
Cosine similarity says:
"Their interests are almost identical."
Dot product says:
"Customer A has a much stronger overall interaction."
Both statements can be useful.
If we're trying to identify similar users based on taste, cosine similarity may be more appropriate.
If we're trying to predict strength of interaction or preference, the magnitude may actually contain valuable information, so a dot product can be preferable.
The Connection to Machine Learning
This distinction becomes even more important when we move from our simple genre example to embeddings.
Suppose a movie is represented by a 768-dimensional embedding:
and a user is represented by:
The dot product is:
Cosine similarity is:
Now the difference becomes very important.
For embedding-based systems such as:
- semantic search
- RAG
- document retrieval
- recommendation systems
- image similarity
- text similarity
we frequently care about whether two embeddings point in similar directions, rather than simply whether their raw values are large.
That's why cosine similarity is so common in embedding-based retrieval.
One Very Important Exception
There is an interesting mathematical shortcut.
If we normalize every vector to length 1, then:
and:
Therefore:
which gives:
So after L2 normalization:
Cosine similarity is simply the dot product of the normalized vectors.
This is an extremely useful connection when you later discuss vector databases and embedding search.
The Final Takeaway
You can summarize the entire concept with this comparison:
| Property | Dot Product | Cosine Similarity |
|---|---|---|
| Measures alignment | ✅ | ✅ |
| Considers magnitude | ✅ | ❌ |
| Measures angle/direction | Indirectly | ✅ |
| Sensitive to vector length | ✅ | ❌ |
| Range | Depends on vectors | -1 to 1 |
| Good for preference strength | ✅ | Sometimes |
| Good for pattern similarity | Sometimes | ✅ |
| Common for embeddings | ✅ | ✅ |
| Same after normalization | — | Yes |
The conceptual difference can therefore be stated very simply:
while:
And for your recommendation-engine example, this gives you a natural progression:
User preferences → Movie features → Dot Product → Problem with magnitude → Cosine Similarity → Normalized embeddings → Vector/embedding search.
That progression will make the mathematics connect nicely to modern recommendation systems, RAG and vector databases.
No comments:
Post a Comment