English
Sign in
Back to Home

Engineering blog

Run LLM calls reliably and cheaply

Technical notes on the cost, reliability and integration of large-model calls. Based on public documentation from Anthropic and OpenAI, combined with what we learned building our gateway.

12articles
11categories
AllCost optimizationIndustry NewsPolicy & IndustryAI InfrastructureAI IndustryIndustry AnalysisAI PolicyIntegration GuideGateway PracticesModel SelectionCost Optimization
Gateway Practices9 min read

429, 5xx, and Timeouts: Engineering Practices for LLM Gateway Failover

Upstream rate limiting and failures are the norm, not the exception, for LLM applications. This article breaks down the correct handling for 429/5xx/timeout/401, when you can automatically switch channels, when you must never retry, and how to avoid double billing.

Source: Admesh 技术团队Read article

Model Selection10 min read

Model Selection and Cost Governance: From 'Always Use the Strongest' to Task-Based Tiering

In the same product, using a flagship model for everything is usually the most expensive approach. This article presents a task-based tiering method for model selection: how to break down tasks, how to use an evaluation set to verify whether downgrading is safe, and how to switch models in the same codebase with a unified gateway.

Source: Admesh 技术团队Read article

Cost optimization8 min read

Batch API or real-time calls: how to choose the right channel for different workloads

Both Anthropic and OpenAI offer batch processing channels, trading a lower unit price for longer response times. This article provides an actionable decision framework: which tasks should go through batch, which must go real-time, and how to manage both types of traffic with the same gateway.

Source: Admesh 技术团队Read article

Cost Optimization9 min read

The Complete Guide to Prompt Caching: How It Works, Cost Model, and Implementation

Prompt caching can reduce the input token cost of repeated prefixes by an order of magnitude. This article breaks down the differences between Anthropic's and OpenAI's cache implementations, which scenarios actually save money, and how to reuse the same caching strategy across multiple models with a unified gateway.

Source: Admesh 技术团队Read article

Start from the docs, integrate in ten minutes

One OpenAI-compatible API, one Base URL and one key for the major models. Pay per token; new accounts get trial credit.

Developer docsSign up free