Your LLM bill needs to come down. How do you know a cheaper setup won’t make the product worse?
Reddit r/MLOps1mo4 min read
Say you’re running an LLM feature in prod and costs are starting to hurt. You find a model/config that’s idk 30% cheaper and it looks basically as good on the prompts you try. What do you actually do before shipping the change? If you’ve been through this in prod, curious what you ended up relying on.. submitted by /u/Alive-Recognition712 [link] [comments]
