I found Ruby LLM to be surprisingly good - in terms of usability it's close to Vercel's AI framework.
It tries to strike a balance between working out of the box and being flexible... which has its challenges, still nice overall.
One big real-life pain I experienced is that caches don't always work, e.g. for xAI, since it only supports completions API and thought signatures are returned wrong.
We use and love RubyLLM! A wonderful and easy to use framework.
Agreed with another commenter on the frustration with the responses API not being naively supported; that seems like a huge miss. There is a connector from another dev, but it's buggy and not as high quality as the main gem.
Really looking forward to future development and especially 2.0!
Edit: Just saw that responses API is now native? I will definitely check that out.
Since a few mentioned Responses API: the reason why it wasn't implemented in 1.x is because RubyLLM 1.x effectively assumes a 1:1 mapping between provider and protocol. That assumption no longer holds since OpenAI has 2 protocols with different capabilities, and to access all VertexAI models we need to support a bunch under that single provider.
Therefore, a major refactoring to split Protocols and from Providers was needed, as well as a way to route different models to different Protocols under the same Provider, transparently.
That's one of the many things that's gonna ship with RubyLLM 2.0.
RubyLLM is very easy to use. Made extensive use of it for a project last year. Drawbacks are it was difficult to instrument for true trace observability and it has a pattern where retries will delete the underlying models so the history you see is clean but not necessarily great for seeing exactly what the sequence of API calls was.
I'm building something that is only pointed at Claude, and I don't anticipate moving away from the Anthropic ecosystem. Does RubyLLM offer me an advantage over directly using Anthropic's Ruby SDK?
To put it differently, is this more like choosing between Fog and aws-sdk-s3, or choosing between Active Storage and aws-sdk-s3?
Why build it in such a way that there you're locked into one provider when you can just use this tool and have the ability to choose any provider in the future? I mean, even for fallbacks, like what if anthropic API is down that particular day when you need the service?
Not the commenter, but the reason is the same as the choice to use direct db driver vs ORM - you get to take full advantage of special features and intricacies of one system, rather than trying to maintain a lowest common denominator of functionality between multiple ones.
You can always build multiple clients however. It could be one thorough integration with one API, and a simpler backup via RubyLLM.
Unless of course RubyLLM supports all of the above, but even then, are they going to be able to keep up as much as a native client in the long term?
The answer is almost always that a native client for a more critical integration is the right call. You can always add another if you need it.
RubyLLM is awesome! I use it on side projects. So interesting how questions and comments from last year SF Ruby conf, https://youtu.be/y535u1EWqAg?si=rbyv52T035apKwQk are already features shipped in the ecosystem.
I spent a lot of time with RubyLLM a few months ago. Very nicely designed and implemented. I have my own LLM clients written in various Lisp languages and I thought about appropriating some of the design of RubyLLM. Imitation is flattery.
We use RubyLLM in production too, the most elegant library in this space I've seen so far.
I also liked how they run the issue tracker. If you select "Feature Request", it makes you explain how you explored workarounds, why you believe it belongs in RubyLLM etc to prevent scope creep.
> Error: You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. * Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 20, model: gemini-2.5-flash Please retry in 41.543129369s.
I have used a personal Gemini token for all users, which often exceeds quota. You can:
1. Check out the repo and use your own tokens
2. Use your own tokens in the browser, they are only held in the browser and are not being persisted or logged in the backend
It is quite nice, but not as nice as you'd want. You still have to set platform specifics when running completions when you want to tune things like temperature, effort, max tokens, etc.
Hi! Valid challenge, I am probably misremembering. We were playing with various 'one-interface to all providers' solutions and I might have mixed up RubyLLM there. Sorry for that.
I will have a deep dive into which things I felt we needed to adapt per provider.
I didn't mean to imply that you have to solve all of our wants of course.
One thing we did do was monkey-patch the spot where tool_calls are performed by RubyLLM. We had our own mechanism for that and were able to skip RubyLLM's and still extract the tool calls and run them through our own tool harness. That all worked beautifully. I don't know if that type of stuff is something you want PRs on or that you want to keep steering towards the route that does everything within RubyLLM classes. Happy to contribute some of that.
We had already implemented tool_calls in our own database and have a system that executes them and creates our conversation array, etc. So we wanted to leverage the providers that RubyLLM supports without having to change the tool execution in our platform.
I have been a fan of Ruby for many years, but in this fast paced era the Ruby ecosystem always struggled with the dependency versioning. Gems I relied on were never available or compatible with the rest of the ecosystem.
Why would anyone still build in dynamically typed languages in 2026? Why relinquish the crystal clear signals that static typing is able to provide to the LLM?
You static typed evangelists have lost your damn minds. You seem to have completely misunderstood what this library even is because you have some primal urge to boast static typing at every chance.
You can build high quality software with dynamically typed languages, and Ruby is an absolute dream to read and write.
Static typed languages had been around a long time before Python, JS and Ruby gained popularity. All three of the latter now support some form of type hints.
Why did people switch to these languages in the first place and what's driving the current back-to-typed-languages trend?
I was on team dynamic typing for about 12 years, and Ruby was a big part of that. I still think dynamic languages can be wonderful to read and write.
But after using modern statically typed languages with good inference, I changed my mind. Many of my old objections were really objections to verbose type systems, not static typing itself. With inference, you can keep a lot of the readability while gaining safer refactoring, better tooling, and earlier feedback.
That doesn’t mean dynamic languages can’t produce high-quality software. They obviously can. But I don’t think appreciating modern static typing is just evangelism.
And yes, I understand what this library is about, it's for "beautiful" easy to use interface to AI providers for Ruby apps. It's the popular play nowadays with litellm, bifrost, gomodel and vercel gateway. We have at least couple AI gateways, libraries like that every week on HN.
Even as rails dev, I am seeing that you might be right. It’s really hard to find specific pros nowadays that Ruby brings to the table. All that talk about conventions over configurations and vast presence of Rails in weights is fun, but if writing speed isn’t an issue anymore, then Ruby on Rails has serious problems with larger codebases
they introduced static typing (Sorbet) to avoid problems, it's completely different app and looks nowhere close to what you experience in standard Rails app
I found Ruby LLM to be surprisingly good - in terms of usability it's close to Vercel's AI framework.
It tries to strike a balance between working out of the box and being flexible... which has its challenges, still nice overall.
One big real-life pain I experienced is that caches don't always work, e.g. for xAI, since it only supports completions API and thought signatures are returned wrong.
Thank you!
Responses API is now implemented and it's coming in RubyLLM 2.0
https://github.com/crmne/ruby_llm/blob/main/lib/ruby_llm/pro...
Do you have any details published around 2.0? Would love to learn more.
Not yet. I'll do a series of blog posts and tweets in the next weeks.
I have an open source gem called Raix that builds on top of RubyLLM's abstractions and is quite popular. https://github.com/OlympiaAI/raix
We use and love RubyLLM! A wonderful and easy to use framework.
Agreed with another commenter on the frustration with the responses API not being naively supported; that seems like a huge miss. There is a connector from another dev, but it's buggy and not as high quality as the main gem.
Really looking forward to future development and especially 2.0!
Edit: Just saw that responses API is now native? I will definitely check that out.
Thank you!
Since a few mentioned Responses API: the reason why it wasn't implemented in 1.x is because RubyLLM 1.x effectively assumes a 1:1 mapping between provider and protocol. That assumption no longer holds since OpenAI has 2 protocols with different capabilities, and to access all VertexAI models we need to support a bunch under that single provider.
Therefore, a major refactoring to split Protocols and from Providers was needed, as well as a way to route different models to different Protocols under the same Provider, transparently.
That's one of the many things that's gonna ship with RubyLLM 2.0.
If you're curious: https://github.com/crmne/ruby_llm/commit/d398354da493570b050... https://github.com/crmne/ruby_llm/commit/0875ce2dfeae9d28a3a...
RubyLLM is very easy to use. Made extensive use of it for a project last year. Drawbacks are it was difficult to instrument for true trace observability and it has a pattern where retries will delete the underlying models so the history you see is clean but not necessarily great for seeing exactly what the sequence of API calls was.
Glad you like it.
Rails-style instrumentation landed in 1.16.0.
https://rubyllm.com/instrumentation/
I'm building something that is only pointed at Claude, and I don't anticipate moving away from the Anthropic ecosystem. Does RubyLLM offer me an advantage over directly using Anthropic's Ruby SDK?
To put it differently, is this more like choosing between Fog and aws-sdk-s3, or choosing between Active Storage and aws-sdk-s3?
Why build it in such a way that there you're locked into one provider when you can just use this tool and have the ability to choose any provider in the future? I mean, even for fallbacks, like what if anthropic API is down that particular day when you need the service?
Not the commenter, but the reason is the same as the choice to use direct db driver vs ORM - you get to take full advantage of special features and intricacies of one system, rather than trying to maintain a lowest common denominator of functionality between multiple ones.
You can always build multiple clients however. It could be one thorough integration with one API, and a simpler backup via RubyLLM.
Unless of course RubyLLM supports all of the above, but even then, are they going to be able to keep up as much as a native client in the long term?
The answer is almost always that a native client for a more critical integration is the right call. You can always add another if you need it.
Laravel has a similar library https://laravel.com/docs/13.x/ai-sdk
RubyLLM is awesome! I use it on side projects. So interesting how questions and comments from last year SF Ruby conf, https://youtu.be/y535u1EWqAg?si=rbyv52T035apKwQk are already features shipped in the ecosystem.
I spent a lot of time with RubyLLM a few months ago. Very nicely designed and implemented. I have my own LLM clients written in various Lisp languages and I thought about appropriating some of the design of RubyLLM. Imitation is flattery.
RubyLLM is great! Been a very happy user for awhile now :)
We use RubyLLM in production too, the most elegant library in this space I've seen so far.
I also liked how they run the issue tracker. If you select "Feature Request", it makes you explain how you explored workarounds, why you believe it belongs in RubyLLM etc to prevent scope creep.
thank you for bringing ruby into AI community and your open-source work. Great language must be explored and get more attention :)
Thank you!
I love how MINASWAN Hacker News is when talking about Ruby!
We use this in production for a few apps. Great project.
I built a similar Ruby based agent development kit that has a different focus and feature set:
https://github.com/tweibley/legate
I have created an open source chatgpt clone with rubyllm, check it out here: https://www.railschat.org/
tried it and got this:
> Error: You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. * Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 20, model: gemini-2.5-flash Please retry in 41.543129369s.
I have used a personal Gemini token for all users, which often exceeds quota. You can: 1. Check out the repo and use your own tokens 2. Use your own tokens in the browser, they are only held in the browser and are not being persisted or logged in the backend
Using RubyLLM in production for https://usetix.io It drives our event chat agent that is enhanced with toolcalls etc. Super happy with it.
It is quite nice, but not as nice as you'd want. You still have to set platform specifics when running completions when you want to tune things like temperature, effort, max tokens, etc.
RubyLLM author here.
I'm not sure where you got that.
`chat.with_temperature(0.2)`
https://rubyllm.com/chat/#controlling-response-behavior
`chat.with_thinking(effort: :high, budget: 8000)`
https://rubyllm.com/thinking/#controlling-extended-thinking
Max tokens is the only one of your list that require provider specific params:
https://rubyllm.com/chat/#provider-specific-parameters
I'm one guy doing it for free. Happy to see your contribution!
Hi! Valid challenge, I am probably misremembering. We were playing with various 'one-interface to all providers' solutions and I might have mixed up RubyLLM there. Sorry for that.
I will have a deep dive into which things I felt we needed to adapt per provider.
I didn't mean to imply that you have to solve all of our wants of course.
One thing we did do was monkey-patch the spot where tool_calls are performed by RubyLLM. We had our own mechanism for that and were able to skip RubyLLM's and still extract the tool calls and run them through our own tool harness. That all worked beautifully. I don't know if that type of stuff is something you want PRs on or that you want to keep steering towards the route that does everything within RubyLLM classes. Happy to contribute some of that.
Interesting! What were you guys trying to achieve by running them in your own tool harness?
We had already implemented tool_calls in our own database and have a system that executes them and creates our conversation array, etc. So we wanted to leverage the providers that RubyLLM supports without having to change the tool execution in our platform.
Well put!!!
You work your arse off for free and the guy who made the disparaging comment didn't even bother to research to see if he had the details right.
Hat's off to you Carmine for all your work. Many people really do appreciate it.
And thank you! It is absolutely awesome and a true joy to work with.
I haven't tried it but it looks promising.
Thank you
In case you're using PHP or Node.js, we've made a similar toolkit free and open source on github: https://github.com/Qbix/AI/tree/main/classes/AI
"What is the best language in the world (say ruby)" ;)
I have been a fan of Ruby for many years, but in this fast paced era the Ruby ecosystem always struggled with the dependency versioning. Gems I relied on were never available or compatible with the rest of the ecosystem.
Why would anyone still build in dynamically typed languages in 2026? Why relinquish the crystal clear signals that static typing is able to provide to the LLM?
You static typed evangelists have lost your damn minds. You seem to have completely misunderstood what this library even is because you have some primal urge to boast static typing at every chance.
You can build high quality software with dynamically typed languages, and Ruby is an absolute dream to read and write.
Static typed languages had been around a long time before Python, JS and Ruby gained popularity. All three of the latter now support some form of type hints.
Why did people switch to these languages in the first place and what's driving the current back-to-typed-languages trend?
Why do you think that follows?
I was on team dynamic typing for about 12 years, and Ruby was a big part of that. I still think dynamic languages can be wonderful to read and write.
But after using modern statically typed languages with good inference, I changed my mind. Many of my old objections were really objections to verbose type systems, not static typing itself. With inference, you can keep a lot of the readability while gaining safer refactoring, better tooling, and earlier feedback.
That doesn’t mean dynamic languages can’t produce high-quality software. They obviously can. But I don’t think appreciating modern static typing is just evangelism.
And yes, I understand what this library is about, it's for "beautiful" easy to use interface to AI providers for Ruby apps. It's the popular play nowadays with litellm, bifrost, gomodel and vercel gateway. We have at least couple AI gateways, libraries like that every week on HN.
Well, LLMs have an obscene amount of context built into their weights about Ruby on Rails, and can work within it extremely quickly.
Even as rails dev, I am seeing that you might be right. It’s really hard to find specific pros nowadays that Ruby brings to the table. All that talk about conventions over configurations and vast presence of Rails in weights is fun, but if writing speed isn’t an issue anymore, then Ruby on Rails has serious problems with larger codebases
Codebases like Stripe?
they introduced static typing (Sorbet) to avoid problems, it's completely different app and looks nowhere close to what you experience in standard Rails app
This is not a tool for using LLMs to write Ruby code.