Basically I have a use-case where I am building an llm-based tutor where some questions you ask the tutor can be adequately answered by gpt-4o-mini but others required gpt-4o. I'm looking for a router that can route to gpt-4o only when necessary and otherwise route to the cheaper gpt-4o-mini. First of all, is this even something that can be done reliably at this point? If so, are there any recommendations for LLM routers that can do this?
The best option I have seen for routing is LiteLLM. Worth a look.
I would check out if Not Diamond (no personal affiliation) [0] fits your use case. They had a really interesting blog post recently where they open sourced their older model router [1].
[0] https://www.notdiamond.ai/features
[1] Blog - Routing on Random Forests - https://www.notdiamond.ai/blog/rorf