Choosing Modelsintermediate13 min · 9 steps
Pricing and Costs
What an LLM request really costs: per-token rates, long-context cliffs, tier multipliers, caching break-even, routing — and when self-hosting actually pays.
request
Key ideaStep 1 of 9
Cost = input tokens × rate + output tokens × rate
LLM APIs bill per million tokens, and output costs more — 5× input on Claude Opus/Sonnet/Haiku 5.5, GPT-6 Sol, GPT-6.1 Terra and Gemini 3.8 Flash. Reasoning tokens bill as output even when you never see them.
A typical RAG request — 500-token system prompt, 2,000 tokens of retrieved context, a 100-token question and a 300-token answer — costs $0.0082 on GPT-6 Sol or Claude Sonnet 5.5 ($2 / $10). The 300 output tokens cost more than half as much as the 2,600 input tokens.
TipAt 10,000 requests a day that’s $82/day, ~$2,460/month on GPT-6 Sol — or $123/month on GPT-6 Luna ($0.10 / $0.50) if quality holds on your evals.
Lesson 3 of 4 in Choosing Models