Embeddings

If you still have a little confusion, don’t worry, we’ll clear it up in this section.

Remember how I asked you to hold on to the thought of vectors?

In your mind, you probably did two things: Remembered the word vector Noted that you need to learn more about vectors

What you did was store that word somewhere in your mind. Then, you embedded meaning along with that word.

In a text representation, that spot in your mind could have looked like this: [“vector = word”, “need = learn more about this”, “reason = this guy told me to remember it”, “confusion level = high”]

That one spot in your mind held all that.

That is how vector embeddings work, only they convert all the text and meaning into numbers.

Now, hold on because it gets a little more interesting.

Those numbers are called dimensions. Think of dimensions like amounts of attributes.

For example, imagine you’re using a color picker tool.

At the bottom is a slider to pick from a range of base colors.

You slide that over to the blues.

When you stop, you now have an area where you can slide around to pick different shades of blue.

When you land on the shade of blue you want, you’re able to copy the RGB value of it. That may look something like this: [52, 61, 235]

The numbers in that list are the amounts of the attributes - Red, Green and Blue.

52 = the intensity of the attribute Red 61 = the intensity of the attribute Green 235 = the intensity of the attribute Blue

If you adjust those numbers, you’ll get different colors.

Still a color, just a different one.

As it relates to a vector database, you can imagine a space in the database for colors. In that space - reds have an area, blues have an area and so on. In those areas, you can slide to different coordinates to get different shades and tints.

In a multi-dimensional vector store, vectors can store many different dimensions.

This makes them very good for “understanding” the abstract relationships of data.

Using the blue color that we chose, we can store concepts like - blue is a “cool” color, the sky is blue, water is blue. There are blue berries. Lots of dimensions.

So, now imagine we ask ChatGPT to give us a list of cool colors, or colors that make us think of water. Vectors give LLMs a way to do that.

Now we have to understand how to put it all together.