Naderu Weekly · Episode 12

Why web search busts your prompt cache

If your AI agent runs a web search on every step, it is not just paying up to fourteen dollars per thousand search queries — it is completely destroying your prompt cache. This week we break down the cost arithmetic of the dual-search split: why live web search and enterprise RAG belong on two completely different pricing models.

Two search jobs, two economic realities

On the Naderu model board and its underlying data file, we officially separated search into two distinct job categories:

• Live Web Search (bb/web-search) — Real-time freshness, open internet indexing, and provider grounding tools. Needed for breaking news, current documentation, and facts outside model pre-training.
• Enterprise RAG (bb/rag-search) — Private vector DB retrieval over internal documentation, code repositories, tickets, and customer records. Bounded inside your VPC with zero external data leakage.

Treating them as the same feature in an autonomous agent loop leads directly to compounding cloud bills.

The search grounding toll booth

When calling a model with live search tools enabled, token pricing ceases to be the whole bill. You pay for input and output tokens plus per-query search surcharges:

• Perplexity Sonar (Agent API) — Per the Agent API documentation and pricing page, tokens bill at $0.25 input / $2.50 output per million. But every web search tool call adds $0.0025 ($2.50 per 1,000 searches), and every URL fetch adds $0.0005.
• Google Gemini 3.8 Flash (Vertex AI Search Grounding) — Sourced to Vertex AI Search Grounding pricing and Gemini Developer API pricing: projects receive 5,000 free grounding queries per month. Past that threshold, Google bills $14.00 per 1,000 search queries on top of base token rates. At three searches per multi-turn answer, search fees exceed $0.04 before counting a single token.
• xAI Grok 4.7 — Sourced to xAI pricing: Grok bills at $2.00 in / $10.00 out per million tokens, with live web tool execution expanding token payloads and latency.

Why live web search kills KV cache reuse

Prompt caching across all major providers — including Anthropic and OpenAI — relies on key-value cache reuse that requires a byte-identical prefix starting from token zero.

When an autonomous agent executes a live web search, dynamic snippets containing live timestamps, volatile news summaries, or shifting search ranking snippets get prepended to the prompt context. Even a single token variance destroys the prompt cache for the entire remainder of the context window. In a 10-step agent loop, every turn bills at full uncached base input rates.

Enterprise RAG: Prompt cache economics

In private enterprise RAG, you control the prompt structure. Placing invariant system instructions, tool schemas, and extraction rubrics first keeps the stable prefix cached:

• Claude Sonnet 5.5 — Caches from 512 tokens. Reads cached tokens at $0.10 per million — a 95% discount off its $2.00 base input rate.
• GPT-6 Luna — Base input is $0.10 per million; prompt cache reads cost just $0.01 per million (a 90% discount) once prompts cross 1,024 tokens.
• Gemini 3.8 Flash — Caches from 4,096 tokens at $0.025 per million across its 1M token context window.

100,000 queries side-by-side

That is a 45x to 75x operational cost difference between live web search grounding and cached vector RAG.

Three moves for Monday