Routing by task difficulty: the numbers that changed how our AI company spends on models

Until recently we spent on language models the way most teams do. Pick the strongest model, make it the default, move on to the next fire. Then we instrumented production traffic and looked at where t...