Embeddings + Search Modernization
Last verified: October 2026· engagement
We replace or augment legacy search (Elasticsearch, Algolia) with semantic search and RAG — often starting as a small AI feature and growing into a retrieval platform.
- Outcome
- A new semantic search / retrieval layer in production
- Timeline
- 6–12 weeks
- Pricing
- $40–120k build + $4–10k/mo ops
- Buyer
- VP Eng at content/doc-heavy companies
The problem
Your users can't find anything. Legacy search (Elasticsearch, Algolia, basic SQL LIKE) misses intent, and the answers are often wrong. Meanwhile, you have embeddings on the roadmap but no clear path from "we want semantic search" to a production retrieval system.
What we do
- Stand up a retrieval layer over your existing data — vector store, hybrid lexical + semantic, the right chunking strategy.
- Ship RAG that cites its sources, so users can trust the answers and your team can debug failures.
- Wire evals into the retrieval loop so quality regressions are caught before they reach users.
- Roll out incrementally — one feature or one surface at a time, with metrics that show the win.
How it fits together
What you get
Built on our open source
ormai — Gives AI agents database access without the risk — scoped, governed access for agents.
Why hybrid retrieval is the production default
Pure semantic search has a predictable weakness that surfaces immediately in production: it is bad at exact tokens. Product codes, error strings, surnames, version numbers, and internal identifiers are precisely what people search for and precisely what embeddings blur, because a model trained on meaning has no reason to keep two similar-looking identifiers apart. Keyword search handles these perfectly. Running both and merging recovers exact-match behaviour without giving up meaning-based recall, which is why nearly every mature system does it and why a proposal to replace keyword search outright is usually a warning sign.
The second thing we tend to find is that teams over-invest in the retriever and under-invest in ranking. Retrieval has two stages: a cheap wide net, then a scoring pass over the candidates. Adding a reranker to an average retriever generally improves results more than replacing an average retriever with an excellent one, because it fixes ordering rather than just recall, and ordering is what the user experiences. It is also far cheaper to change later.
Chunking is where most of the quality actually lives, and it is the least glamorous part. Chunks that split mid-argument return fragments that are individually retrievable and collectively useless; chunks that follow the document's own structure, carry their heading context, and overlap slightly at the boundaries retrieve as coherent answers. We evaluate this the same way we evaluate anything else — a set of real queries with known good answers, scored before and after each change — because retrieval quality is otherwise argued about on anecdote.
When this is not the right engagement
Retrieval work is worth it when the content is right and the results are not. It is a poor fit when:
- Your search is failing because the content is wrong or missing. No retrieval layer fixes a corpus problem, and it is worth checking first.
- Query volume is low enough that a well-tuned keyword index with synonyms would do. Semantic retrieval is not automatically an upgrade.
- There is no way to judge whether results improved. Without a labelled query set this becomes a matter of opinion, and we will ask you to build one.
Common questions about this engagement
Do you work under NDA?
Yes — we sign your mutual NDA before any data or repo access. For audits we prefer read-only access to start; for builds we work in a clean repo under your ownership.
Will we own what you ship?
Always. You own the code, the runbooks, the dashboards. We are explicitly set up to hand off and transition out, not to create dependency.
Vendor-agnostic — what does that mean in practice?
We integrate with what you already run — OpenAI, Anthropic, open-weight models on your cloud, your CI/CD, your observability stack. If a hosted vendor solves it, we will not reinvent it; if a self-hosted tool is the right answer, we will not pretend the hosted one is.
How is this different from a Big Four consulting deck?
We are the engineers doing the work, not analysts handing recommendations to a different team. The deliverable is working software in your repo, not a slide deck.
Can you work with our in-house AI team instead of replacing them?
Yes — most of our engagements pair with an internal owner and ramp them up to run the system after we leave. Many of our best engagements start with "we hired an AI team, help us get them productive."
Will semantic search replace our keyword search?
Almost certainly not, and a proposal to replace it outright is a warning sign. Pure semantic search is bad at exact tokens — product codes, error strings, surnames, version numbers — which are precisely what people search for. Running keyword and semantic retrieval together and merging recovers exact matching without giving up meaning-based recall, which is why nearly every mature system is hybrid.
How do you know whether results got better?
With a labelled query set, built before anything changes. A few dozen real queries with known good answers, scored before and after each change, is enough to turn a debate about whether search feels better into a measurement. Without one this work becomes a matter of opinion, and we will ask you to build the set as the first task — including when it delays the interesting engineering, because everything after it depends on it.
Guides for this work
What is RAG (retrieval-augmented generation)?
RAG retrieves the passages relevant to a question and puts them in the prompt, so the model answers from your data with citations instead of relying on what it happened to memorise during training.
LLM evalsHow to build LLM evals
Evals turn 'the model feels better' into a number you can gate a deploy on — they are the test suite that makes every prompt, model, and pipeline change safe to ship.
Let’s scope it on a call
Thirty minutes with an engineer. We’ll tell you straight whether this is the right first move for your team.