AI agents failing obvious tasks: retrieval bugs, not reasoning — Anthropic's 49% and 67%

An agent that can summarise a long PDF, call an API correctly and follow a multi-step workflow will still miss the refund window buried in a policy document, grab the wrong product SKU or forget a tool result it received ten seconds earlier. In a post republished on DEV Community, developer Lars Winstand argues that most "the agent is dumb" bugs are not reasoning failures at all — they are failed fetches, and that changes where debugging should start.

The failure looks like reasoning. It isn't

The pattern Winstand describes is familiar from production support bots and internal copilots: the agent handles the hard parts, then misses one painfully obvious step. It feels like bad reasoning, but usually one exact fact was missing at one exact moment. "That's not a reasoning failure. That's failed fetch," he writes.

Anthropic's numbers: 49% and 67% fewer retrieval misses

The argument rests on Anthropic's Contextual Retrieval research: when chunks are contextualised and combined with semantic search plus Contextual BM25, failed retrievals drop by 49%. Adding a reranking step pushes the reduction to 67%. Standard RAG — basic chunking with pure semantic retrieval — cites no comparable figure. As Winstand puts it, a lot of agent unreliability "is not model IQ; it's underbuilt retrieval".

Long context and vector search are not enough

Two common assumptions fall apart. First, handing the model the whole documentation does not guarantee it uses the right passage: because of the "Lost in the Middle" effect, models often do worse when the relevant fact sits in the middle of a long prompt, so an agent can retrieve a fact and still bury it. Second, pure embedding search loses exact-match fights — order IDs, policy titles, product SKUs, workflow names, error codes and ticket IDs. Hybrid retrieval, combining keyword search, semantics and reranking, keeps winning in real systems.

Debug the retrieval layer first

Winstand's checklist starts with logging the exact context the agent saw — retrieved chunks, prior messages, tool outputs, system prompt and injected memory — because "if you can't inspect the final prompt state, you're debugging blind". Then separate session memory, durable memory and retrieval, since they are three different incidents. Add keyword search for literal terms, rerank before swapping models, and test where the critical fact sits in the prompt.

The last item is architectural: if a knowledge base fits comfortably in context — roughly under 200,000 tokens — Anthropic suggests skipping RAG altogether, and notes that prompt caching can cut latency more than twofold and cost by up to 90%. His rule when an agent fails an obvious task: ask what retrieval returned, where the fact appeared, whether exact-match search existed, whether reranking ran, and whether the agent saw the right thing in a usable form.