Embeddings explained
An embedding is a list of numbers representing a piece of text, produced so that similar meanings land close together.
The useful intuition
Imagine placing every sentence on a map where position encodes meaning. "How do I reset my password" and "I forgot my login details" end up near each other, despite sharing almost no words. "How do I reset my router" ends up somewhere else.
Real embeddings do this in hundreds or thousands of dimensions instead of two, but the intuition holds.
What this buys you
Keyword search fails when the user's words differ from the document's words. Someone searching "cannot sign in" gets nothing from a page titled "Authentication troubleshooting".
Embedding search finds it, because the meanings are close even though the strings are not. That is the whole reason this technique is worth its complexity.
How the search works
- Embed every chunk of your content once, ahead of time. Store the vectors.
- When a question arrives, embed the question.
- Find the stored vectors closest to it, usually by cosine similarity.
- Return those chunks.
A vector database does step three efficiently. At small scale, say under a few thousand chunks, an array and a loop is genuinely fine. Reach for infrastructure when you have measured a need for it.
Practical rules
Use one model for everything. Vectors from different embedding models are not comparable. Change the model and you must re-embed everything.
Embedding is cheap, generation is not. Embedding costs a small fraction of what generating an answer costs. Re-embedding a corpus is an affordable operation.
Store the text alongside the vector. You cannot turn a vector back into text. If you lose the original, the vector is useless.
Where it breaks down
Exact identifiers. Searching for error code E4021 or SKU AB-9931 works badly. These carry no semantic meaning, and near matches are actively wrong.
Negation. "Deployments that did not fail" and "deployments that failed" embed close together. The vector does not reliably encode the not.
Numbers and dates. "Under fifty pounds" and "over fifty pounds" look similar to an embedding model.
Jargon the model has not seen. Internal codenames have no meaningful position on the map.
The fix is usually hybrid
Combine embedding search with keyword search and merge the results. Keyword search nails exact identifiers; embedding search handles paraphrase. Together they cover each other's failures.
Most production systems that work well do this. Pure vector search is the common first build and the common first thing to outgrow.
Next
Chunking and indexing covers how to split content up, which affects retrieval quality more than the choice of embedding model does.