Start with billed traces, not a token-count guess
A useful LLM price comparison needs separate input and output distributions. System prompts, retrieved documents, tool schemas, conversation history, image or audio input, reasoning tokens, and retries can be invisible in the product UI but visible on the invoice. Sample real traces across short, median, and long sessions; do not multiply one happy-path prompt by monthly users.
Use the calculator for a blended scenario, then run high and low cases. If a router offers multiple underlying providers, pin the provider or preserve routing metadata during the benchmark. Otherwise a price, latency, or data-policy change can be caused by the route rather than the model.
Cache and batch are workload properties
Prompt caching helps only when the provider recognizes a stable prefix and the request meets its cache rules. Measure actual cache-read and cache-write usage instead of assuming every repeated system prompt is discounted. Cache storage or write meters may exist, and a short-lived prefix may never repay its setup cost.
Batch pricing belongs to evaluation runs, document processing, enrichment, moderation backlogs, and other delay-tolerant jobs. It does not belong to an interactive chat path merely because the batch rate is lower. Split the architecture into real-time and asynchronous queues, then compare each workload on its own service objective.
- Record exact model IDs and snapshots instead of relying on a moving latest alias.
- Capture provider usage fields and invoice line items for reconciliation.
- Treat fallback requests and safety retries as separate billable attempts until proven otherwise.
- Compare rate limits, latency percentiles, regions, retention, and structured-output behavior after cost.
Choose the cheapest viable route, not the cheapest row
A low token rate can lose money if output quality creates more human review, prompts must be longer, tool calls are unreliable, or rate limits force unused reserved capacity elsewhere. Define the pass condition first: task success, latency, schema validity, citation accuracy, safety, language quality, and operational availability. Cost is evaluated only among routes that pass.
For a migration, dual-run a bounded sample and log both responses without exposing customer data. Compare accepted-task cost, not just token cost. Keep the integration behind a provider adapter so routing, authentication, and provider-specific parameters do not leak across the product.
Common comparison mistakes
- Adding input and output token prices without weighting the actual token mix.
- Applying a cache discount to tokens that do not meet the provider cache contract.
- Treating batch as a drop-in replacement for an interactive endpoint.
- Ignoring reasoning, tools, search, media, retries, and fallback requests.
- Selecting a moving model alias without tracking the resolved model version.
Questions developers ask
Which LLM API is cheapest?
The cheapest viable route is the one that passes your quality, latency, policy, and availability gates at the lowest measured cost per accepted task.
Should I include batch API pricing?
Yes only for work that can tolerate the documented asynchronous turnaround. Keep real-time and batch workloads as separate cost scenarios.
How do I validate this estimate?
Run a bounded production-like sample, preserve billed usage fields and provider request IDs, then reconcile the calculator with the invoice before scaling.