2 points | by mihir_ahuja a day ago ago
2 comments
LLMuxer automates cost–accuracy trade-offs across LLM providers through OpenRouter, so you can easily benchmark and compare dozens of models with one API key.
!pip install llmuxer llmuxer run --dataset banking77_test.json --models openai:gpt-4o-mini openai:gpt-4o claude:claude-3-5
LLMuxer automates cost–accuracy trade-offs across LLM providers through OpenRouter, so you can easily benchmark and compare dozens of models with one API key.
!pip install llmuxer llmuxer run --dataset banking77_test.json --models openai:gpt-4o-mini openai:gpt-4o claude:claude-3-5