How Much Does RAG Cost at 50,000–100,000 Requests a Month?
The number of requests alone does not determine the price. A retrieval-augmented generation (RAG) system searches your documents for relevant passages and passes them to a model before it answers. The bill therefore depends on input and output tokens, model calls per question, retries, and the services around the model. In the illustrative example below, the model portion for 100,000 monthly requests ranges from $100 to $2,000 with the same request size but different models.
Keep two figures separate. Model charges are token fees that can be calculated. Total cost of ownership also includes retrieval, document indexing, infrastructure, quality monitoring, and staff time. To build a useful budget, measure your request profile, apply current rates, and calculate several traffic scenarios, including costs that do not appear on the API invoice.
What happens inside one user request
The user sees one question and one answer, while the system may perform several paid steps:
- Query rewriting. A model turns a conversational question such as “what about a return if I’ve already opened the box?” into a search query. This is a separate, usually short call.
- Query embedding. The text is converted into a numeric vector used to find similar passages. This has its own model and rate.
- Retrieval. A vector database or search service finds candidate passages. You pay for the service or infrastructure, not model tokens.
- Reranking. A separate model may reorder the retrieved passages by relevance.
- Answer generation. The instructions, retrieved passages, conversation history, and question go in; the answer comes out. This is often the most expensive step.
- Validation. Another call may check whether the answer is grounded in the documents or addresses a disallowed topic.
- Retries. Network errors, rate limits, or invalid output formats can trigger another call.
Not every system uses every step. A simple RAG flow may combine retrieval with one generation call; a more complex one may use three or four model calls per question. For a budget, count both calls per question and their size, rather than requests alone.
A transparent token-cost formula
Tokens are pieces of text processed by a model. Input and output tokens are generally priced separately, and output can cost several times more.
Cost per call = input tokens ÷ 1,000,000 × input rate + output tokens ÷ 1,000,000 × output rate.
Monthly model cost = cost per call × calls per question × questions per month.
When a provider supports caching, repeated input may use separate rates for regular input, cached reads, and cache writes. In the illustrative Standard, short-context rates checked on 11 October 2026, gpt-6.1-sol charged $2 per million regular input tokens, $0.10 for cached reads, and $2.50 for cache writes. A write cost more than regular input. Do not assume the entire prompt is cached; use actual usage records to separate the volumes.
Do not mix service tiers or context lengths. A batch rate does not apply to a chatbot that must answer immediately, and a short-context price may no longer apply when a request exceeds the provider’s threshold. In the same Standard tier, the long-context rates used here for gpt-6.1-sol were $4 per million input tokens and $15 per million output tokens.
A calculation for 50,000 and 100,000 requests
This is an educational example, not a forecast for your system or a BotLabs quotation.
Assumptions: one model call per question; 6,000 input tokens (instructions, retrieved passages, and question) and 800 output tokens; no caching, retries, or taxes. The example uses USD API rates checked on 11 October 2026 for Standard service and short context: gpt-6.1-sol at $2 per million input and $10 per million output tokens; gpt-6-luna at $0.10 and $0.50.
For one gpt-6.1-sol call: 6,000 ÷ 1,000,000 × $2 = $0.012; 800 ÷ 1,000,000 × $10 = $0.008; total $0.02. For gpt-6-luna: $0.0006 + $0.0004 = $0.001.
| Model | Cost per call | 50,000 questions | 100,000 questions |
|---|---|---|---|
| gpt-6.1-sol | $0.02 | $1,000 | $2,000 |
| gpt-6-luna | $0.001 | $50 | $100 |
A 20-fold difference does not show that the cheaper model is the better choice: this example does not compare quality. A price comparison is meaningful only if both models produce acceptable answers on your data.
Now add illustrative monthly costs of $150 for retrieval and hosting, $100 for monitoring, and $600 for maintenance: $850 beyond model charges.
| Model | 50,000 questions including $850 | 100,000 questions including $850 |
|---|---|---|
| gpt-6.1-sol | $1,850 | $2,850 |
| gpt-6-luna | $900 | $950 |
The $850 is neither a market benchmark nor a service price. It simply shows how recurring operating costs can exceed the token bill for a lower-cost model.
Next, change the architecture while keeping the rates constant. First rewrite the query (1,000 input and 100 output tokens), then generate an answer with a longer context (10,000 input and 1,000 output tokens), and retry 5% of generation calls. For gpt-6.1-sol, rewriting costs $0.003, generation $0.03, and expected retries $0.0015: $0.0345 per question, or $3,450 for 100,000 questions, versus $2,000. For gpt-6-luna, the total is $0.001725 per question, or $172.50, versus $100. That is a 72.5% increase at the same volume and rates. Without a real request profile, a single “RAG costs X” estimate tells you little.
Other recurring costs
Model charges are only part of the monthly budget. Include vector search and the managed service or your own database; embeddings for new or updated documents; storage for source files, chunks, and logs; monitoring for errors and latency; backups and a fallback provider or degraded mode; quality evaluation; and maintenance of integrations and the knowledge base.
Quality evaluation also takes resources. You may separately check whether retrieval found relevant context, whether the answer is supported by that context, and whether the answer itself is useful. Automated evaluation may require additional model calls; manual sampling takes staff time. Neither is included in the token rate.
One-time costs and what can increase the bill
Treat knowledge-base preparation, CRM or ticketing integrations, data cleanup, and an initial set of test questions as one-time work, separate from monthly operations. If you allocate initial costs across months, state the period. That keeps operating cost distinct from the time needed to recover the initial investment.
Model scenarios that can raise spend: longer answers increase output tokens; longer conversation histories and more retrieved passages increase input; errors and limits cause retries; crossing the short-context threshold may change the rate; and traffic peaks may require more capacity.
Monthly volume does not show concurrent demand. For illustration, 100,000 monthly requests across 22 working days is about 4,545 a day. If 15% of daily volume arrives in the peak hour, that is roughly 682 an hour, or 11–12 a minute. For an API, check your account’s rate limits. For a self-hosted server, test whether it can handle the load at an acceptable latency. A load test should account for arrival rate and concurrent connections, not just the monthly total.
How to build your own budget
- During a pilot or from a historical sample, measure average and 90th-percentile input and output tokens at each step, calls per question, retries, and cached tokens.
- Define acceptable answer quality, maximum latency, and what the system should do when it does not know.
- Apply the rate for the correct service tier and context length, and record the date checked.
- Calculate at least three volumes—such as 50,000, 100,000, and 150,000 questions—plus a longer-answer and retry scenario.
- Add recurring costs and list one-time expenses separately.
- Assign an owner to compare the estimate with actual spend every month and adjust settings.
For each system step, record calls per question, input, cached and output tokens, the rate and tier, the date checked, and monthly cost. Add separate rows for retrieval, storage, monitoring, evaluation, and maintenance. Empty cells reveal missing data. Mark estimates clearly and compare them with actuals later.
Takeaway and first steps
There is no single fixed price for 50,000–100,000 RAG requests. Context size, calls per question, retries, and rates change the model bill; retrieval, quality checks, and maintenance add to the total. For a lower-cost model, costs around the API can exceed token charges.
Start by mapping the steps in one answer, measure tokens at each call, separate model charges from total cost, calculate several traffic levels, and compare models on the same test set. Check rates again before making a budget decision. BotLabs can discuss your inputs and identify what to measure in a pilot; any estimate depends on the actual process profile.
Frequently asked questions
They may differ in context size, model, calls per question, answer length, retry rate, peak load, retrieval method, and maintenance. In this educational example, adding query rewriting, a longer context, and a 5% retry rate increased model charges by 72.5% at the same rates and volume.
No. The API bill covers model token processing. Knowledge-base updates, quality checks, retrieval fixes, integrations, storage, monitoring, and backups are separate expenses unless included in another service fee.
Only if it meets your quality requirements. A model that makes more errors may require retries, human escalation, and rework. Compare models on the same test set and account for the cost of an accepted result and the cost of mistakes, not just the cost of one call.
RAG budget consultation
In 15 minutes with a BotLabs technical specialist, discuss one process, expected demand and the costs of retrieval, answer evaluation and maintenance. Identify the missing inputs for a total RAG cost model and what to measure in a pilot.
Process and requests
Total cost
15 minutes with a specialist to outline RAG cost items and the inputs for an initial estimate.
