use cases · support bot, same model ·
A support assistant on GPT-6 Luna costs €54 a month for 30,000 conversations. We kept the model and changed only how the assistant uses it: how it sends the conversation, its tools, and when it calls the model at all. Quality is graded on the same 180 questions. (How Luna was chosen is another use case.)
59% less, same answers
Most of it from one change: the conversation is now cached, so OpenAI reads it at a tenth of the price instead of billing it in full at every message. Not calling the model for thank-yous and after a hand-off does most of the rest.
the clever tricks lost money
Routers, cheaper models for easy messages, a shorter knowledge base, less reasoning with stricter instructions: each cost more than it saved, or cost quality.
GPT-6 Luna, used better · 30,000 conversations a month
new bill−59%
−44% from caching alone
The clever tricks lost money: routers and cheaper models for easy messages.
Each change saves on top of the ones above it. The faint part of each bar is what that change took off the bill.
same model
| change | saves | a month, after it | score |
|---|---|---|---|
| As first set up the setup picked in the model test | 98.0 | ||
| Conversation cached OpenAI reads it from its cache at a tenth of the price instead of billing it in full at every message | −€23.8 | 98.3 | |
| One calculator for dates and amounts fewer back-and-forths with the model | −€1.1 | 98.3 | |
| Fixed reply to a thank-you “You’re welcome!” without calling the model | −€3.0 | 98.3* | |
| No model call after a hand-off a person has the conversation from then on | −€3.9 | 98.3* |
| part | a month | share |
|---|---|---|
| Conversation so far re-read at every message, mostly from cache | 42% | |
| Knowledge base and tools read from cache, at a tenth of the price | 22% | |
| Reasoning | 22% | |
| Replies already short: 36 words at the median | 14% |
| idea | a month | what happened |
|---|---|---|
| Less reasoning | −€4.9, −€3.1 | Scores 92.0 and 94.8 instead of 98.3: wrong totals and deadlines, legal threats not passed to a person. |
| Less reasoning, with stricter instructions | −€1.4 | Score 96.0. To stay accurate it calls its calculator more, and every call is billed for the whole conversation again. |
| A router sending easy messages to less reasoning | +€0.7 to +€2.5 | The router costs €1.30 to €3.09 a month and saves €0.61: easy messages (thanks, OK) hardly reason anyway. |
| Sending easy messages to a cheaper model | about €0 | Once the conversation is cached, DeepSeek V4.1 Flash is only 25% cheaper a message, switching loses the cache, and it scores 95.5. |
| A shorter knowledge base | −€0.8 | Scores 94.1 and 94.0: shortened rules became ambiguous. |
| Other cache settings | €0 to +€18 | OpenAI's automatic caching already covers it all. The worst, a separate cache per customer, bills the knowledge base again for every conversation. |
make it cacheable
Send the conversation so that its beginning never changes. Rewritten into one new message each time, it never hits the cache.
one cache for everyone
A separate one per customer bills the shared knowledge base again for every conversation.
skip known answers
A thank-you, a conversation a person has taken over: that was 24% of our messages. Check yours.
less reasoning is a false saving
It costs quality, or comes back as more calls.
Synthetic data: the same 180 questions, in English and French, as the model test. A month is 30,000 conversations at OpenAI's list prices. No client data.
The thank-yous (13% of messages) and the messages after a hand-off (12%) are our scripts' shares, not real traffic. Costs vary about 10% from one run to the next.
Full study on request: hugo@jayn.app
*on your own data and at your volume. you only pay her if she generates savings.
the cheapest option
that meets your quality bar, and its bill.
what to fix first
before switching, so it does not cost you customers.
your own model
whether it pays, and from what volume.