How we cut LLM inference cost by 35% without touching quality
Model routing, caching, and context compression — in the order they paid off. No heroics, mostly plumbing.
2 posts
Model routing, caching, and context compression — in the order they paid off. No heroics, mostly plumbing.
We gate every prompt and retrieval change behind an eval suite, the same way we gate code behind tests. Here's the setup, and what it caught.