In this category I’m a big fan of https://github.com/bentoml/BentoML
What I like about it is their idiomatic developer experience. It reminds me of other Pythonic frameworks like Flask and Django in a good way.
I have no affiliation with them whatsoever, just an admirer.
Superb product and team.
Worth looking into if you’ve done any engineering work around deploying ML models as a or within a service.
Looks interesting, what if I need to write some logic (pre/post prediction) in the prediction server?
Looks great! What is the argument to use this over MLFlow model packaging and serving?
This is likely to share its name with the next Prime Minister of the UK...
This looks promising. It feels like for non ML engineers it’s very hard to figure out how to use models as part of vanilla CRUD codebase.
For instance in a Rails app the ML model services would probably be served as a completely external service API generated with something like Truss wrapped in a service class that just exposes the outputs and handles errors/input validation!