What a customer-support bot actually costs per conversation
Support volume is measured in tickets. Turn the ticket into tokens, and the model choice becomes a visible line in the support budget.
A support bot is the most common first AI feature, and the most commonly mis-budgeted one. The per-token rate you compare models on is not the number the finance team sees; the number they see is cost per conversation, and it is dominated by how much context you resend, not by which model answers.
The token profile comes first
For the arithmetic below, one support conversation is modelled as 3,000 input tokens and 450 output tokens: a 600-token policy and tone prompt, a 300-token knowledge snippet per turn across three turns, the user's messages, and answers around 150 tokens each. That is a deliberately ordinary profile — the point is to see which part of it you control.
What that costs across the current dataset
At 10,000 conversations a month — a mid-sized support desk — the six cheapest blended-cost models in the index price out as follows. The cache column assumes 70% of the input is a stable prefix served from cache, which is realistic for a policy-and-knowledge prompt.
| Model | Provider | Input /1M | Output /1M | Cost / month | With 70% cache |
|---|---|---|---|---|---|
| Gemini Exp 1114 | Google Gemini | free | free | free | no cached rate |
| Gemini Exp 1206 | Google Gemini | free | free | free | no cached rate |
| Gemma 3 27b It | Google Gemini | free | free | free | no cached rate |
| Gemma 4 26b A4b It | Google Gemini | free | free | free | no cached rate |
| Gemma 4 31b It | Google Gemini | free | free | free | no cached rate |
| Learnlm 1.5 Pro Experimental | Google Gemini | free | free | free | no cached rate |
The four numbers you control
- Prompt length. A 600-token system prompt resent on three turns is 1,800 tokens of the 3,000. Trim it, or cache it.
- Retrieved context. Send the paragraph that answers the question, not the whole article. Retrieval precision is a cost decision as much as a quality one.
- History policy. Full transcripts grow every turn. A rolling window or a summary of earlier turns keeps input flat instead of growing.
- Answer length. Output is billed at a higher rate than input on essentially every model here — median output rate ${usd(medianOutput)} against median input ${usd(medianInput)} — so a terse answer style is a permanent discount.
What to exclude from the business case
Free tiers, committed-use discounts and regional rate cards sit outside these rates, and a support bot also has non-token costs (retrieval infrastructure, evaluation, human escalation). The token line is the one this index can measure; keep the rest in a separate column so the two are never confused.
Change the request volume, the token profile or the cache rate and re-price the whole index.
Model your own support workloadFrequently asked
› How many tokens is one support conversation?
A three-message exchange with a 600-token system prompt, a 300-token knowledge snippet per turn and a 150-token answer typically lands near 3,000 input and 450 output tokens. Change any of those four numbers and the estimate changes with them — which is why the article shows the arithmetic rather than a single figure.
› Why is my support bot bill higher than the per-token price suggests?
Two reasons: the system prompt and retrieved context are counted on every turn, so input tokens grow faster than the conversation does, and the transcript is resent as history on each turn. Both are cache-shaped problems.
› Would a cheaper model fix an expensive support bot?
It moves the bill, but cache handling and prompt trimming usually move it further, and they do not risk answer quality. Start with what you resend, then with which model answers.
Where the numbers come from
The data pages this article draws on — check any figure against them, they are recomputed on every refresh.