Day 2 | What are vectors | Vector Similarity metrics | Vector Database Zero to Hero

Summary

The video introduces vectors as the basic numerical representations used in machine learning and vector databases. It then explains two ways to compare them: cosine similarity and Euclidean distance.

1. What vectors are

• In machine learning, a vector is a one-dimensional array of numbers.
• Each number represents a feature of an object, so vectors are also called feature vectors.
• For example:
  • [180] could represent a car’s speed.
  • [180, 15] could represent its speed and mileage.
  • [180, 15, 30000] could add its price.
• A vector’s number of dimensions corresponds to the number of features it contains. More features produce higher-dimensional vectors.
• A vector is different from:
  • A scalar, which is one number.
  • A matrix, which is a two-dimensional array.
  • A higher-dimensional array, such as a three-dimensional array.
• A vector is not necessarily an embedding. Embeddings are a particular kind of vector, discussed later in the series.

2. Dimensions and similarity in vector space

• With one feature, objects can be placed on a number line. With two or three features, they can be represented in 2D or 3D space.
• In the car example, cars with similar speeds, mileage, and prices would be close together, while cars with different features would be farther apart.
• Real-world models can use many dimensions—far more than can be visualized. The same basic intuition applies: vectors that are similar are often closer or more aligned in the vector space.
• Similarity metrics provide a numerical way to compare two vectors.

3. Cosine similarity

Cosine similarity compares direction, not vector size or magnitude. It is based on the cosine of the angle between two vectors:

• 1 means the vectors point in the same direction.
• 0 means the vectors are perpendicular (orthogonal).
• −1 means the vectors point in opposite directions.

The formula is the dot product of the vectors divided by the product of their magnitudes.

For example, the vectors [2, 0, 1] and [4, 0, 2] have the same direction, even though one is twice as large. Their cosine similarity is approximately 1.

The presenter says cosine similarity is widely used for comparing embeddings because direction is often more important than magnitude. Mentioned applications include search, recommendations, and retrieving relevant context in RAG systems.

4. Euclidean distance

Euclidean distance measures the straight-line distance between two vectors, so magnitude and position affect the result.

To calculate it, subtract corresponding components, square the differences, sum them, and take the square root. Its range is 0 to infinity:

• 0 means the vectors are at the same point.
• Larger distances indicate that the vectors are farther apart.

For example, each component of [3, 1, 2] and [6, 4, 5] differs by 3. The distance is:

[ \sqrt{9 + 9 + 9} = \sqrt{27} \approx 5.2 ]

The video names GPS, robotics, and other situations involving physical distance as applications.

5. Choosing a metric

• Use cosine similarity when the direction of vectors matters more than their magnitude.
• Use Euclidean distance when the actual distance between points matters.
• The appropriate metric depends on the problem and its context.

The video concludes that similarity metrics are foundational to semantic search, recommendation engines, and AI systems.

Speakers and sources

• Speaker: One unidentified presenter or narrator; no other speakers are featured.
• Sources: No external sources are identified in the subtitles.