RAG Is Simpler Than You Think: Six Retrieval Architectures Without Over-Engineering

Why retrieval gets over-engineered

Most teams building retrieval-augmented generation (RAG) jump straight to embeddings, vector databases and reranking pipelines, writes the Lighthouse Newsletter, while their users simply want to find the document that explains how to reset a password. There is a right tool for each problem in engineering, the author argues, and retrieval systems are no exception.

Before picking an architecture, the piece lists five factors to weigh: how fresh the data must be, how large and how volatile the corpus is, what kind of queries arrive, the expected query volume, and the team's own machine-learning experience.

Six recipes, from minimal to elaborate

The first recipe is plain full-text search: BM25, Elasticsearch or PostgreSQL's built-in search. Queries cost nothing, results come back in under ten milliseconds, no model deprecation can break it, and it needs neither a chunking strategy nor an evaluation set. It does miss synonyms and cannot handle conversational "how do I..." questions.

The second is agentic query rewriting, where a language model turns a messy question into clean keywords, drops stopwords, adds synonyms and splits complex requests. The stated cost is roughly $0.001 per query with a small model, and improving results means editing a prompt rather than re-embedding a corpus.

Hybrid search follows: BM25 selects 50 to 100 candidates that an embedding model reranks down to ten. At text-embedding-3-small pricing of $0.02 per million tokens, embedding 50 documents of about 500 tokens each costs roughly $0.0005 per query, or about $15 a month at a thousand queries a day. The real expense is latency, with 200 to 500 milliseconds added per query.

The remaining recipes are on-the-fly embedding for high-churn corpora, a hot/cold split that pre-embeds only the 20 percent of documents receiving 80 percent of traffic, and full pre-embedding for very large, stable corpora. Pre-embedding a million documents costs about $10 once, plus $10-30 a month for storage, but switching embedding models later costs $10,000 when everything is pre-embedded, $2,000 for a hot tier and nothing at all on the fly.

Questions with several intents

Real users ask compound questions such as reading a CSV file, cleaning the missing values and plotting the result, which are three separate intents. Systems such as Perplexity decompose them, route each sub-query to the cheapest adequate method and synthesise a single answer. The newsletter's arithmetic puts that at $0.002 per query against $0.03 without decomposition.

The bottom line

The suggested rule of thumb: about 60 percent of systems should stop at full-text search plus query rewriting, 25 percent need a hybrid with on-the-fly or hot/cold embedding, and only 10 percent justify full pre-embedding. Start with the simplest option, measure your own search for two to four weeks, and move down the list only when the data says you must.