Ain't Nothing But A G Thang

October 9, 2009

You've probably seen more than a few articles on the web showing how to build a Rack app. If not, here's a good one to start with. You'll quickly see that building a Rack app is really simple, which is why Rack is awesome, because it's simple. But what about writing a Rack-compliant server? Well it turns out that is pretty easy as well.

I just pushed a little Rack-compliant HTTP Server that I wrote using GServer to github. The whole thing is less than 200 lines of code. The core of it is short enough that I can explain how it works here.

First, GServer. GServer, which is short for "Generic Server" makes it pretty simple to create a multithreaded TCP Server. Taking out some error handling code, here's what the GServer looks like for our Rack HTTP Server:

module GThang
  class HttpServer < GServer

    attr_reader :port, :rack_app

    def initialize(options={})
      @port = options[:Port] || 8080
      @rack_app = options[:rack_app]
      super(@port)
    end

    def serve(socket)
      RackHandler.new(socket, rack_app, port).handle_request
    end

  end
end

So all there is to a GServer is basically a serve method. This will be called each time a client connects to the server. The argument to the method is the client socket connection. You read and write data from the socket as you see fit for your application. As you can see here, we just pass the socket, along with the rack app and the port to the RackHandler initializer and then call handle_request on that. We'll look at how you setup the rack app in a minute, but first let's take a look at the meat of what the RackHandler does. The handle_request method looks like this:

def handle_request
  return unless add_rack_variables_to_env
  return unless add_connection_info_to_env
  return unless add_request_line_info_to_env
  return unless add_headers_to_env
  send_response(app.call(env))
end

So what happens is the various add_ methods build up the rack environment. Once the environment is ready, we call the rack app. The rack app responds with the standard 3 element array, which we pass off to the send_response method, which writes the actual http response to the client. Take a look at the full code for this on github for the details.

Now the fun part is that we now have a fully functional HTTP server that is capable of acting as a file server or serving a Rails app. All we have to do is give the HttpServer the correct Rails app. If you look in the examples, you see this for the file server:

GThang::HttpServer.run(
  Rack::CommonLogger.new(
    Rack::Lint.new(
      Rack::Directory.new(root, Rack::File.new(root)))),
  :Port => 8080)

Now I choose to write it this way to make it clear what is actually happening. You will normally see the builder DSL used to configure a rack app, which would look like this:

use Rack::CommonLogger.new
use Rack::Lint.new
use Rack::Directory.new(root)
run Rack::File.new(root)

This is obviously a lot cleaner, but to understand how Rack works, you have to realize that all this is doing is what we see in the first example. A Rack app with Rack middleware is simple a chain of apps that call the next app in the chain, possibly modifying the environment or response before or after the rest of the chain is called.

So there you have it, beauty in simplicity.

Posted in Technology | Tags Rack, Ruby, Rails | 8 Comments

Why Rails 3 Will Require Ruby 1.8.7

September 14, 2009

This past weekend I attended the Windy City Rails conference. It was in a great location in the heart of downtown Chicago and seemed to have a pretty good turn out. There were many great talks but this blog post will be focusing on a specific talk, and more precisely, part of a talk. Yehuda Katz gave a talk on the status of Rails 3. One of the things that he mentioned, which you may have already heard, is that Rails 3 will require Ruby 1.8.7 or higher, dropping support for Ruby 1.8.6. He also mentioned why they are doing this and I found the reason to be interesting. It's not that the Rails core team wants to try to take advantage of any specific new features, it's that Ruby 1.8.6 has a bug which has been fixed in 1.8.7.

To see the bug in action, I recommend that you install Ruby Version Manager (rvm). Once you have installed rvm, install Ruby 1.8.6 and Ruby 1.8.7.

The bug is that in Ruby 1.8.6, the hash method for Hash doesn't generate the same hash code for different hashes with the same values:

$ rvm use 1.8.6
$ irb
ruby-1.8.6-p383 > {:x => 1}.hash
 => 1313270 
ruby-1.8.6-p383 > {:x => 1}.hash
 => 1307060 
ruby-1.8.6-p383 > {:x => 1}.hash
 => 1296440 
ruby-1.8.6-p383 > {:x => 1} == {:x => 1}
 => true
ruby-1.8.6-p383 > h = {{:x => 1} => "foo"}
 => {{:x=>1}=>"foo"} 
ruby-1.8.6-p383 > h[{:x => 1}]
 => nil

So despite the fact that two hashes have the same values and are equal, you can't use a hash as a key in a hash, because that depends on the hash codes of the values being equal, which they aren't. This is fixed in Ruby 1.8.7:

$ rvm use 1.8.7
$ irb
ruby-1.8.7-p174 > {:x => 1}.hash
 => 327875 
ruby-1.8.7-p174 > {:x => 1}.hash
 => 327875 
ruby-1.8.7-p174 > {:x => 1}.hash
 => 327875 
ruby-1.8.7-p174 > {:x => 1} == {:x => 1}
 => true 
ruby-1.8.7-p174 > h = {{:x => 1} => "foo"}
 => {{:x=>1}=>"foo"} 
ruby-1.8.7-p174 > h[{:x => 1}]
 => "foo" 

This is important because you could use a hash cache calls to a method that expects a hash, but only if you can use a hash as the key. This is one of the main reasons Rails 3 is going to require 1.8.7. They could make it worth for both 1.8.6 and 1.8.7 and higher, but why? It simplifies things to just require that you upgrade to Ruby 1.8.7 to use Rails 3. If you are using 1.8.6, this is probably a gotcha that you should be aware of.

Posted in Technology | Tags Ruby, Rails | 0 Comments

Infinite Recursion

September 2, 2009

A few days ago I posted an article on Tail Call Optimization. One really quick way to determine if any language/VM/interpreter performs Tail Call Optimization (TCO) is to write a function that calls itself, in other words, creating an infinitely recursive function. If the function just runs forever and doesn't return, the interpreter is doing TCO, otherwise you will get some sort of stack overflow error. So I decided to test a variety of languages to see what happens when you write an infinitely recursive function. First up is ruby:

def forever
  forever
end

forever

It was no surprise to me that running this results in this error:

run.rb:2:in `forever': stack level too deep (SystemStackError)
    from run.rb:2:in `forever'
    from run.rb:5

Ruby doesn't have TCO, I knew that. Next up, Python:

def forever(): forever()

forever()

This has a similar result, but the stack track is a little more revealing:

Traceback (most recent call last):
  File "run.py", line 3, in <module>
    forever()
  File "run.py", line 1, in forever
    def forever(): forever()
  File "run.py", line 1, in forever
    def forever(): forever()
...
  File "run.py", line 1, in forever
    def forever(): forever()
  File "run.py", line 1, in forever
    def forever(): forever()
RuntimeError: maximum recursion depth exceeded

This shows pretty clearly what is going on, the stack frames are pilling up and eventually it gets to the point where the Python interpreter says enough is enough. Interestingly enough, this mailing list thread shows that it's completely feasible to add TCO to Python and Guido just doesn't want to add it.

JavaScript is no surprise either, but we can write our infinitely recursive function with an interesting little twist:

(function(){ arguments.callee() })()

Yes that's right, it's an anonymous recursive function! Running this with SpiderMonkey results in InternalError: too much recursion. So how about Java:

class Run {
  public static void main(String[] args) {
    Run run = new Run();
    run.forever();
  }

  public void forever() {
    forever();
  }
} 

The stack trace for this one looks a bit like the Python one, we see a stack frame for each iteration:

Exception in thread "main" java.lang.StackOverflowError
    at Run.forever(Run.java:8)
    at Run.forever(Run.java:8)
    at Run.forever(Run.java:8)
    at Run.forever(Run.java:8)
...

So that means no TCO on the JVM. So Scala doesn't have TCO, right?

def forever : Unit = forever

forever

Wrong. This one runs forever. Scala does TCO with some bytecode tricks which Nick Wiedenbrueck explains really well in this blog post.

Clojure is a functional Lisp saddled with the problem of no-TCO on the JVM, but it gets around it in a slightly different way than Scala. If you write a tail recursive function like this:

(defn forever [] (forever))

You will get a java.lang.StackOverflowError just as you do in Java. Instead, Clojure provides a language level feature to do recursion:

(defn forever [] (recur))

Calling recur in the function makes it call the function again. recur also checks that it is in the tail recursive position, which ends up being an interesting feature because you have to explicitly say "I expect this to do TCO". In other languages that do it transparently, you might think TCO is happening, but it might not be and you won't find out until you get a stack overflow error. Also, this construct allows us to have recursive anonymous functions:

((fn [] (recur)))

Erlang, being a function language, has TCO as well:

-module(run).

-compile(export_all).

forever() ->
  forever().

Now one thing all of these functional languages that have TCO; Scala, Clojure and Erlang, all have in common is that while the infinitely recursive function runs, it essentially does nothing, but it will peg your CPU utilization to nearly 100%. This next language, Haskell, being the mother of all functional langauges, of course has TCO. Here's the function:

forever = forever

Yes, that's a function. And if you call it with forever, it just sits there and runs forever. But here's the crazy thing, it uses no CPU. My guess is that it has to do with lazy evaluation, but I'm not sure. Any ideas?

Posted in Technology | Tags TCO, Javascript, Scala, Haskell, FunctionalProgramming, Python, TailCallOptimization, Ruby, Java, Erlang, Clojure | 19 Comments

Active Record Random

August 30, 2009

Are you looking for a way to select a random record that plays nice with named scope in your Rails app? Throw this into config/initializers/active_record_random.rb:

class ActiveRecord::Base
  def self.random
    if (c = count) > 0
      first(:offset => rand(c)) 
    end
  end
end

Now you can call random the same way you would call first or all. That means you can do Widget.random to get a random widget or Widget.fancy.random, to get a random fancy widget, assuming you have defined the fancy named scope for Widget.

Posted in Technology | Tags NamedScope, RubyOnRails, Ruby, Rails | 6 Comments

In Search Of Sharper Tools

July 21, 2009

After reading the comments in my last post, one thing I realized that I neglected to do is define what I mean by metaprogramming. Rubyists probably already know what I mean, but people coming from other programming languages might have different ideas about what I mean by metaprogramming.

I've actually mentioned this in a few talks I've given about Ruby. I don't really like the word Metaprogramming, it's a little bit nebulous. I think a better term is dynamic code generation. I like that term because I think most programmers will have a pretty good mental model of what I am talking about when I say that. There are several features of Ruby that when combined together allow you to do almost anything to bend the language to your will. To understand how to do that, I recommend reading the book Metaprogramming Ruby, which is in beta right now.

I'll give a short, real world example of what I'm talking about. I'm working on a Rails app that uses a database that has bit field columns. I want to treat the boolean flags in the bit field as though they are regular boolean attributes throughout my code. So I wrote a Ruby gem has-bit-field, which generates all the methods necessary to work with the bit field columns. You define what is in the bit field in the most clear, simple, elegant way possible by just stating which column has the bit field and then what attributes should be generated for each bit:

class Person < ActiveRecord::Base
  has_bit_field :bit_field, :likes_ice_cream, :plays_golf, :watches_tv, :reads_books
end

This is the kind of abstraction that the metaprogramming capabilities of Ruby afford you. I threw this together, with tests, in an hour or so. Can you imagine the amount of nonsense you would have to go through in Java to create an abstraction equivalent to this?

This type of abstraction is what I think makes Ruby a great language, but I realize you are definitely walking a fine line with this kind of thing. It's possible for this sort of thing to go wrong quickly. The first way these things go wrong is when the abstraction is not intuitive and not documented. First of all, a good abstraction should be almost intuitive, requiring other programmers to be able to guess what is does. This is commonly referred to as the Principal of Least Surprise. This doesn't mean that you are excused from providing some sort of documentation explaining how it works, especially for more complex abstractions.

The reason why it's important that the abstraction is clear is that most of the time the code that defines the abstraction is dense at best, and downright ugly at worst. This isn't a problem specific to Ruby, as anyone who has worked with Lisp macros can attest to. But in the end I'd rather have a small chunk of code that is tested and documented that I don't really need to look at that enables me to make the code where the business logic is defined as clear as possible. If other programmers are constantly having to dive into the guts of the definition of these abstractions just to understand how the code works, you have officially created a mess. And this is no ordinary mess, this is meta-spaghetti, and is a mess on an order of magnitude not possible in statically typed languages.

So does this mean you shouldn't use Ruby? Not at all, and I think Glenn Vanderburg sums it up best:

Weak developers will move heaven and earth to do the wrong thing. You can't limit the damage they do by locking up the sharp tools. They'll just swing the blunt tools harder.

I think developers often associate "blunt tools" with static typing, because really they associate static typing with Java. I'm not sure that static typing is in fact a blunt tool. If static typing means I can't create these kinds of abstractions, then yes, it's a blunt tool. But can you do this kind of thing with Scala Compiler Plugins? How about with Template Haskell? What about with MetaOCaml? If you can, are those tools then sharper than Ruby? Or is there a way to define abstractions like these without metaprogramming at all?

Posted in Technology | Tags Scala, DynamicTyping, MetaOCaml, OCaml, Haskell, StaticTyping, Metaprogramming, Ruby, Java | 0 Comments

<< Newer Articles   Older Articles >>