Model Selection and Cost Governance: From 'Always Use the Strongest' to Task-Based Tiering

Why 'Always Use the Strongest Model' Is a Cost Trap

Model capabilities are tiered: flagship models excel at long-chain reasoning, complex code, and multi-constraint instructions; lightweight models are often already sufficient for classification, extraction, format conversion, and short Q&A. Yet the unit price gap between the two can reach ten times or more.

Anthropic's current pricing page and OpenAI's [API pricing](https://developers.openai.com/api/docs/pricing) both divide their models into several tiers; taking Anthropic's published [pricing](https://claude.com/pricing) as an example, the Opus, Sonnet, and Haiku tiers target tasks of different intensity, with obvious unit price differences. Handing hundreds of thousands of simple classification requests per day to a flagship model is the most easily overlooked waste.

Implement Task Tiering in Code

1. List the LLM calls in the system by 'task type' (classification / extraction / summarization / generation / reasoning / code).
2. Choose a default tier for each task type and put it in configuration, rather than model names scattered everywhere.
3. Prepare 50–200 fixed evaluation samples for each task type, including typical hard cases.
4. Run a baseline with the default tier first, then try downgrading and compare accuracy / format compliance rate.
5. Keep the downgrade only when the quality loss is below the threshold; otherwise return to the previous tier.

Stepped Routing: Cheap First, Expensive Later

A practical pattern is 'cheap first, expensive later': process with a lightweight model first, and escalate to the flagship model when the following signals appear—

• The output cannot be parsed (broken JSON, missing fields).
• The model itself declares uncertainty, or confidence is below the threshold.
• A business-side quality check is triggered (for example, the generated SQL cannot be executed).
• The request's complexity metrics exceed the threshold (input length, number of referenced documents).

This way most traffic stays on the cheap tier, and only the truly difficult parts pay for the flagship tier. Note that escalation counts should be included in monitoring: if the escalation rate stays high for a long time, it means the model selection should be re-evaluated.

Things to Confirm Before Cross-Vendor Switching

Different vendors are not consistent in output style, tool-calling format, and refusal policies for the same task. Before switching a task type from vendor A to vendor B, at least confirm:

• Structured output: whether the support level for JSON mode / strict schema is consistent.
• Tool-calling format: parameter structure, parallel calls, and error return methods.
• Context length and truncation behavior: whether overly long input errors out or is silently truncated.
• Content policy differences: the same input is refused by one model and works normally on another.

Tiering Within the Same Gateway

Admesh maintains OpenAI-compatible protocols externally, and models are exposed in the form of namespace/model (for example, anthropic/…, openai/…, deepseek/…). This means task tiering only requires maintaining a 'task type → model' mapping table in configuration, and the code always only changes the model parameter.

Because routing only occurs within the channel pool of the same public_model, it will not silently replace your anthropic/xxx with another vendor's model—tiering is your explicit choice, not something the gateway decides for you.

View the model list API →