use cases · support bot, right model ·
An AI assistant answers the customers of a company that rents out bikes, in English and French: prices, pauses, cancellations, repairs, theft. On Claude Opus 5.5, one of the best models to date, it costs €3,524 a month. We tested the cheaper options, down to an open model on our own GPU. One does as well, for 2% of the price.
a test drawn from the business
180 realistic customer conversations, including the ones that need a person to step in.
every option, graded and priced
Every reply graded to a human's standard, every option priced at 30,000 conversations a month.
a bike rental's customer chat · 30,000 conversations a month
new bill−98%
€41,600 saved a year, on one assistant
Cheap models are not interchangeable: MiMo, at the same €54, made 16 errors.
The same help content, tools and 180 conversations for every option. Real errors are checked by hand.
seven options
| option | a month | real errors, of 180 | against Opus today |
|---|---|---|---|
| Claude Opus 5.5 Anthropic, today | 2 | the reference | |
| GPT-6 Luna OpenAI | 0 | As good, for 2% of the price | |
| Claude Haiku 5.5 Anthropic | 2 | As good, for 3% of the price | |
| DeepSeek V4.1 Flash open model, hosted | 5 | A few more errors, 1% of the price | |
| MiMo-V2.6-Flash open model, hosted | 16 | 1 reply in 11 wrong | |
| GLM-5.3-Flash open model, hosted | 25 | 1 reply in 7 wrong | |
| Qwen3.8-27B open model, on your own GPU | 8 | Costs more, 157% of the price |
not interchangeable
At the same €54 a month, Luna made no real error and MiMo made 16. Only a test on your own cases finds the right one.
fix the content first
Luna needed the help content fixed first: before that, it made 6 real errors in 180. Better to find a model's gaps before your customers do.
yours is not this one
Here the policies were clear. Refunds, a messy help centre or a strict brand voice can change the ranking.
your data, your contract
Luna runs on OpenAI's servers: check where your customers' data goes, and under which terms, before switching.
Synthetic data: 180 conversations in English and French with the customers of a fictional company. The same help content and tools for every option, every reply graded. No client data.
List prices of 10 October 2026 at $1.12 to the euro, five customer messages a conversation.
Full study on request: hugo@jayn.app
*on your own data and at your volume. you only pay her if she generates savings.
the cheapest option
that meets your quality bar, and its bill.
what to fix first
before switching, so it does not cost you customers.
your own model
whether it pays, and from what volume.