Embeddings
Last verified: June 2026· definition
An embedding model converts a piece of text into a vector — a list of numbers — such that semantically similar texts produce nearby vectors. This lets you measure meaning by distance rather than keyword overlap.
Embeddings power semantic search, clustering, deduplication, and the retrieval step in RAG. You embed your documents once, store the vectors, and at query time embed the question and find the nearest documents.
Choice of embedding model, chunk size, and the vector store all shape retrieval quality — which in turn shapes the quality of anything built on top, including RAG answers.
Chunking decides more than the model does
Teams tend to agonise over which embedding model to use and then split documents on a fixed character count. That is backwards. A chunk that splits a table from its header, or a clause from the sentence establishing its subject, produces a vector representing something nobody would ever search for. Chunking on document structure — sections, headings, natural boundaries — usually improves retrieval more than moving to a stronger embedding model.
Embeddings find similar, not relevant
Vector similarity measures whether two passages are about the same thing. It does not measure whether a passage answers a question, and those come apart more often than expected. A document repeatedly discussing a topic without stating the specific fact you need will still rank highly. This is the main reason production retrieval usually combines vector search with keyword search and a reranking pass rather than relying on similarity alone.
Changing the model means rebuilding everything
Vectors from different embedding models are not comparable, and there is no conversion. Switching models requires re-embedding the entire corpus and rebuilding the index, which on a large corpus is a real project with real cost. Because model choice is expensive to reverse, it is worth evaluating candidates against your own queries before committing, and worth storing the source text so a rebuild is always possible.
Metadata belongs with the vector
Retrieval in production almost always needs to respect something beyond similarity: which tenant owns a document, whether the user may see it, how recent it is, whether it is still current. Storing that metadata alongside the vector, so it can be filtered during search rather than after, avoids the common failure where the top results are all filtered away and the user gets an empty page. It is also the main practical reason teams keep vectors in their existing database rather than a separate store.
Common misconceptions
MythA higher-dimensional embedding is a better embedding.
RealityDimensionality is a design choice, not a quality ranking. Larger vectors cost more to store and search, and frequently perform no better on a specific corpus. Retrieval quality on your own queries is the only measure that settles it.
MythEmbeddings understand the text.
RealityThey encode distributional similarity learned from training data. That is enough for semantic search and often reads as understanding, but it carries no notion of truth, recency, or authority — a confidently wrong document embeds just as well as a correct one.
MythOnce documents are embedded, retrieval is solved.
RealityEmbedding is the first step. Chunk boundaries, hybrid keyword search, reranking, and how many results you pass to the model all shape final answer quality at least as much, and are where most retrieval debugging time actually goes.
Frequently asked questions
What chunk size should I use for embeddings?
Prefer structural boundaries over a fixed number. Split on sections, headings, or logical units so each chunk is a coherent, self-contained piece of meaning. Where a fixed size is unavoidable, overlap consecutive chunks so a fact spanning a boundary appears whole in at least one of them. Then measure recall on real queries — chunking is empirical, and the right answer varies by corpus.
Do I need a dedicated vector database?
Not necessarily. Postgres with a vector extension handles a great many production workloads and keeps your vectors alongside the relational data you are usually filtering on anyway, which removes a synchronisation problem. Dedicated vector databases earn their place at large scale or when you need features they specialise in. Starting with what you already run is rarely the wrong call.
Why does my semantic search return topically related but useless results?
Because that is what vector similarity optimises for. The usual fix is hybrid retrieval — combine vector search with keyword search so exact terms, names, and identifiers are not lost — followed by a reranking model that scores each candidate against the actual query. Reranking in particular tends to produce a larger improvement than any amount of embedding-model tuning.