RubyLLM: A delightful Ruby way to work with AI

There’s a whole world of async IO in Ruby that doesn’t get enough attention.

Checkout the async gem, including async-http, async-websockets, and the Falcon web server.

https://github.com/socketry/falcon

Thank you for your kind words!

Valid point. I'm actually already working on testing better streaming using async-http-faraday, which configures the default adapter to use async_http with falcon and async-job instead of thread-based approaches like puma and SolidQueue. This should significantly improve resource efficiency for AI workloads in Ruby - something I'm not aware is implemented by other major Ruby LLM libraries. The current approach with blocks is idiomatic Ruby, but the upcoming async support will make the library even better for production use cases. Stay tuned!

From https://rubyllm.com/#have-great-conversations

    # Stream responses in real-time
    chat.ask "Tell me a story about a Ruby programmer" do |chunk|
      print chunk.content
    end

I’ve found the Ruby community really cares about DUX. Not sure why it’s not in other language communities

I was an early contributor to Langchain and it was great at first - keep in mind, that's before chat models even existed, not to mention tools, JSON mode, etc.

Langchain really, I think, pushed the LLM makers forward toward adding those features but unfortunately it got left in the dust and became somewhat of a zombie. Simultaneously, the foundational LLM providers kept adding things to turn them more into a walled garden, where you no longer needed to connect multiple things (like scraping websites with one tool, feeding that into the LLM, then storing in a vector datastore - now that's all built in).

I think Langchain has tried to pivot (more than once perhaps) but had they not taken investor $$ early on (and good for them) I suspect that it would have just dried up and the core team would have gone on to work at OpenAI, Anthropic, etc.

langchain and llamaindex are such garbage libraries: not only they never document half of the features they have, but they keep breaking their APIs from one version to the next.

Thank you! This is what the Ruby community has always prioritized - developer experience. Making complex things simple and joyful to use isn't just aesthetic preference, it's practical engineering. When your interface matches how developers think about the problem domain, you get fewer bugs and more productivity.

Would anyone happen to know of a similar library with as good DX but for JavaScript or TypeScript?

IMO, the samples look great because they're ridiculously simple.

It doesn't deal with any of the hard problems you'll routine face with implementation.

What about it is a breath of fresh air? What do the other libraries do that this doesn't?

Thanks for flagging this. The eval was only in the docs and meant only as an example, but we definitely don't want to promote dangerous patterns in the docs. I updated them.

bobby drop table, still a thing

I think it's the very nice-looking and clean high-level API that should be a pleasure to use (when it fits the job, of course).

I'm pretty sure this API semantics (instance builder to configure, and then it's ask/paint/embed with language-native way to handle streaming and declarative tools) would look beautiful and easy to use in many other languages, e.g. I can imagine a similar API - save, of course, for the Rails stuff - in Python, C# or Erlang. While this level of API may be not perfectly sufficient for all possible LLM use cases, it should certainly speed up development time when this level of API is all that's possible needed.

Is it really Ruby or they just made a nice interface? I don't see why a hypothetical TypeScript example would be all that different.

    // Just ask questions
    const chat: Chat = LLM.chat;
    chat.ask("What's the best way to learn TypeScript?");
    
    // Analyze images
    chat.ask("What's in this image?", { image: "ts_conf.jpg" });
    
    // Generate images
    LLM.paint("a sunset over mountains in watercolor style");
    
    // Create vector embeddings
    LLM.embed("TypeScript is powerful and scalable");
    
    // Let AI use your code
    class Calculator {
      description = "Performs calculations";
      params = {
        expression: { type: "string", desc: "Math expression to evaluate" },
      };
    
      execute(args: { expression: string }): string {
        return eval(args.expression).toString();
      }
    }
    
    chat.withTool(new Calculator()).ask("What's 123 * 456?");

been using https://github.com/alexrudall/ruby-openai for years with no issues which is a fine gem and works great.

Ruby has a bunch of tools to interact with LLMs already. Also has had bindings to stuff like Torch and Tensorflow for years.

What about it is so thoughtful?

That's my takeaway from Ruby - syntax matters, and good syntax makes programmers happier

I support software in many languages and ruby has to be the most syntactically stimulating.

You're confusing beautiful with simple. There's a lot of complexity and magic that's hidden behind the curtains of that "beautiful" syntax. Great for scripts and small programs, and an absolute nightmare on large projects. It's too simple.

In Ruby, chat = RubyLLM.chat is a method call since Ruby doesn't have properties, only methods. Dropping parentheses is standard Ruby style, familiar to Ruby developers. While adding parentheses is allowed, it doesn't match Ruby's readability. The library aims for a clean style consistent with Ruby conventions.

Because everything in Ruby is an object and everything is a passing messages (ie a method call).

saw this the other day: https://github.com/activeagents/activeagent

haven't tried it yet though

That’s an example of what your code could look like. For example, you might have a Rails app with a Document model and have added search to that model via Searchkick. Then this code is called by the library to execute a search.

From searching the codebase, I can only find eval() used in the markdown and the specs.

You’re totally right that eval()’ing unknown code is terrible but it doesn’t look like the gem itself is doing that.

The usage of eval() is in a user written tool in the docs. Definitely a bd example and should probably be changed

What's a good way to learn the modern Ruby ecosystem nowadays?

I played with Ruby when I was a teenager (~2015 or so), and I definitely remember enjoying it. I know there's still a vocal group of users who love it, so I would be interested in digging in again.

In terms of LLM code generation as well, the well structured nature of a Rails application, where there is a place for everything, a structure for tests to be added, really helps from the perspective of getting a comprehensible application out of it that is easy to modify. In addition to the existence of well tested component for most normal web application tasks, maybe it helps that a lot of Rails has already been based on old-fashioned code generation for 20 years.

Absolutely love this approach. The power and versatility Rails provides for data modeling is top notch. You're 100% right that many of the problems you must solve when building a SaaS app are the same.

There’s a lot of folks who get immense schadenfreude talking about things they know nothing about to strangers on the internet who also don’t know anything.

Don’t let it bum you out.

Ruby, specially with Rails, is particularly suited for AI coding, because of how mature it is and convention over configuration: Most of the important stuff is up to date in the model, and the entire thing comes with a fairy comprehensive set of ideas of how to to be used cohesively to built entire apps.

It's worth remembering that the trolls that complain about Ruby do so because they care about it.

You'll often see the same names coming back on every post to angrily insist that no one is interested in Ruby

...apart from them obviously because if they didn't they would be busy trolling something else. :P

> some trollish comments about no one using ruby anymore

It is somewhat objectively true:

https://octoverse.github.com/2022/top-programming-languages

https://github.blog/wp-content/uploads/2024/10/GitHub-Octove...

It doesn't mean much, and this library can be reproduced in any of those top 10 languages from what I can tell.

Surely you can have the same API elegancy and overall semantics in Python?

    chat = python_llm.Chat()
    _ = chat.ask"What's the best way to learn Python?")

    # Analyze images
    _ = chat.ask("What's in this image?", image="python_conf.jpg")

    # Generate images
    _ = python_llm.paint("a sunset over mountains in watercolor style")

    # Stream responses
    for chunk in chat.ask("Tell me a story about a Python programmer"):
        print(chunk.content)

    # Can be a class if necessary, but for this weather thingy we can probably do with a simple callable
    # Requires Python 3.9+ for typing.Annotated
    def get_weather(
        latitude: Annotated[Decimal, "Latitude of the location"], 
        longitude: Annotated[Decimal, "Longitude of the location"]
    ) -> str:
        """
        Gets current weather for a location.
        """
        ...

    _ = chat.with_tool(get_weather).ask("What's the weather in Berlin? (52.5200, 13.4050)")

Ruby has significantly more capable metaprogramming facilities, but they aren't used in RubyLLM, it's all just objects and methods (biggest difference being use of iterable in Python vs providing a block in Ruby, as I felt an iterable would be more Pythonic here), which is nothing Ruby-specific.

And IMHO advanced metaprogramming should be used carefully, as it may make code pretty but really hard to comprehend and analyze. My largest issue with Rails is difficulty to tell where things are coming from and what's available (lack of implicit imports and ability to re-open any class or module and inject more stuff in there so there's no single place that defines it is a double-edged sword that may lead to chaos if wielded carelessly - YMMV, of course, I'm merely stating my personal preferences here).

at least 2 ways.

1. Moderators can re-submit interesting stories from a second chance pool https://news.ycombinator.com/pool (This might happen automatically from time to time?) When this happens some of the timestamps get updated but others dont.

2. Moderators can invite users via email to re-submit stories. https://news.ycombinator.com/invited

The threads have presumably been merged; look at the submitters in the history.

> 166 points|ksec|4 days ago|21 comments

> 168 points by ksec 4 hours ago

But yes; it’s a bit dodgy to resurface old news like this imo and pretend it’s new news.

I’d go as far as to say that being at #1, under the circumstances, means it’s been artificially boosted somehow.

I haven’t the foggiest why anyone would bother though.

It's not that slow really. Also, there's yjit included https://speed.yjit.org/

We're running a few very popular services on rails and it's really not a problem in any way.

It's not. The method, RubyLLM.chat creates a new instance of the Chat object, as has been copied here:

  module RubyLLM
    class Error < StandardError; end

    class << self
      def chat(model: nil)
        Chat.new(model: model)
      end

If you're wondering about module RubyLLM. That's just how Ruby is often written.

Addendum: Ruby does not require you to put the opening and closing parenthesis on a function to run that function, and it's not always put there when you have zero or 1 parameter (I find it to be cleaner when you have a parameter, but have no opinion when there isn't a parameter)

In the example code from the link itself, you'll see:

  chat.ask "What's the best way to learn Ruby?"

  chat.ask("What's the best way to learn Ruby?")

What's fishy about wrapping a bunch of APIs?

https://ruby-concurrency.github.io/concurrent-ruby/1.1.5/Con...

https://thoughtbot.com/blog/my-adventure-with-async-ruby

First two hits on Google.

Ruby isn't dead and like all alive things, it grows and changes; your expectation is out of date.

https://www.ruby-lang.org/en/news/2020/12/25/ruby-3-0-0-rele...