Semantic Routers: Cutting Claude Code Tokens by 456x
How semantic routing reduces Claude Code context consumption by 456x — from 228K tokens to just ~500 per task. Full analysis of the progressive disclosure problem, embedding mechanics, and practical tips for AI agent developers.
The Skill Catalog Problem — Why 4,556 Skills Don't Fit in a 200K Context Window
Claude Code skills (and Cursor's, GitHub Copilot's, and every other AI agent framework's skills) ship as simple markdown files in a folder. Each skill has a name and brief description stored in the file's frontmatter. When an agent starts, it uses a "progressive disclosure" strategy — every skill's name and description are loaded into the system prompt at startup, while the full body is only loaded when the agent decides to invoke a particular skill.
This approach works fine for small skill collections but breaks down catastrophically at scale. Progressive disclosure handles the body problem — you don't pay for skill bodies you never use — but it does NOT handle the index problem. Even just the names and descriptions are loaded for every skill, every single session, before any work has even started.
Here's what the math looks like in practice:
| Skills in Catalog | Tokens (names + descriptions) | Share of 200K Context |
|---|---|---|
| 100 | ~5K | 2.5% |
| 500 | ~25K | 12.5% |
| 1,000 | ~50K | 25% |
| 4,556 | ~228K | OVERFLOW |
Even when the catalog physically fits, attention quality degrades. Past about 1,000 entries, the agent starts making wrong picks. There's also no garbage collection — nobody removes stale skills, nobody flags duplicates, the catalog only grows. The semantic router pattern solves all of these problems.
How Semantic Router Works — The Technical Breakdown
The semantic router pattern decouples the catalog from the prompt entirely. Each skill's name and description is stored once in an embedding index — a numeric vector representation that captures the semantic meaning of the text. When a task comes in, the agent runs a single semantic search call against the task description, finds the top-5 most similar skills by cosine similarity, picks the most relevant one, and reads the full body only for the one skill it selected.
Think of it like a librarian who has memorized the titles of every book (traditional approach) versus one who has a smart card catalog system (semantic router). The first approach requires the librarian to carry the entire catalog in their head. The second lets them look up only what they need, when they need it.
Test Results: 456x Token Reduction With 87.5% Accuracy
The empirical test results are compelling. Out of 4,556 total skills in the community corpus, 686 were randomly sampled and indexed into mesh-memory using a single sentence-transformer model (intfloat/multilingual-e5-base, 768-dimensional vectors). Eight diverse task queries were then run against the index:
| Metric | Result |
|---|---|
| Strict Top-1 Accuracy | 62.5% |
| Top-5 Cluster Accuracy | 87.5% |
| Query Latency | Sub-second |
| Tokens per Task | ~500 vs ~228K (456x reduction) |
Why This Matters for AI Agents
For agencies like SiTech building agent-based services, this pattern is transformative. Agent efficiency depends directly on context management — and semantic routing makes context usage predictable regardless of how many skills you have. The practical implications are significant: lower token costs, faster response times, better accuracy on skill selection, and the ability to scale to thousands of skills without degrading performance. At enterprise scale, these savings translate to real dollars — from approximately $34,200/day in token costs (at full 200K context) down to roughly $75/day with semantic routing.