Module 0 · Lesson 8 of 10
Embeddings: meaning as numbers
Search by meaning (closest 3)
- I was charged twice for the same order this month.closeness 0.90 out of 1, an example score
- My package arrived damaged and I need a refund for it.closeness 0.86 out of 1, an example score
- The item does not match the listing, so I'd like a reimbursement.closeness 0.84 out of 1, an example score
Search for the same words
No sentence shares a word with "I want my money back".
Closest by meaning: I was charged twice for the same order this month.; My package arrived damaged and I need a refund for it.; The item does not match the listing, so I'd like a reimbursement..
- Signing in
- Refunds
- Delivery
- Closing an account
What this simplifies. Real embeddings have hundreds of numbers, not two, and a model computes them; this map was placed by hand to show the idea. Real systems usually compare two lists of numbers by the angle between them, but the idea is the same: close means similar in meaning. "Search for the same words" is also a crude stand-in. It matches exact word forms of four or more letters, with no ranking, so "refund" does not match "refunds".
Step through one search
The map above shows the result. This shows the work behind it: the text becomes a vector, and the vector is compared with every stored ticket.
Every ticket the index returns is shown.
Predict first: how many tickets come back with these settings? Step through to check.
- User — healthy
- Application — runs each step — healthy — NOW — has the question
- Embedding model — text to vector — healthy
- Vector index — 24 ticket vectors — healthy
- Text
- Answer
- Search by vector
Step 1 of 8 · Application
Done: The user asks a question
- What happens
- The user types a question: "I want my money back"
- Why
- The application holds 24 old support tickets. It wants the ones that mean the same as this question.
- What changes
- The application has the question as text. Text cannot be compared by meaning yet.
- Next
- The embedding model turns the text into a vector
Event log
7 more to go. Press a line to go back to that step.
The map: each ticket's vector as a place
- Signing in
- Refunds
- Delivery
- Closing an account
Closest tickets
The question has no vector yet, so nothing can be compared.
Step 1 of 8: Application. The user asks a question. The user types a question: "I want my money back"
Things to try
- Pick "Can you check on my delivery and process my refund?". It asks about two things. See which one the search finds.
- Pick "How do I close the delivery window?" and ask for 5 tickets. One of them is about something else.
- Keep that question and set the minimum closeness to 0.85. Count how many tickets are left.
- Set the minimum to 0.90 and try each question. Find one where nothing is shown.
What this simplifies. A real embedding model works out hundreds of numbers for a text. Here each place was set by hand and has two numbers. The score is an example made from the distance on this map; real systems often use the angle between two vectors. This index scores all 24 tickets. A real index holds millions of vectors and skips most of them, so it can miss a close one now and then. A good minimum score depends on the embedding model and has to be found by testing.
An (A list of numbers that stands for the meaning of a piece of text. Texts with similar meanings get similar lists.) model turns a piece of text into a list of numbers called an embedding. It does not write answers. Many apps call a separate embedding model for search, apart from the model that writes answers. The exact set-up depends on the system.
Texts with a similar meaning get embeddings that land close together, even when they share no words at all. Texts with different meanings get embeddings that land far apart.
This lets software compare meaning with plain arithmetic: measuring the distance between two lists of numbers, instead of matching exact words. It says nothing about whether either text is true. Close also means the same topic, not the same claim: "refunds are allowed" and "refunds are not allowed" can land close together.
In system design: Your backend calls an embedding model once per (A short passage cut from a longer document, so it can be turned into an embedding and found on its own.), a short passage cut from a document. It calls it again for each question, then compares the numbers.