Paul Barry

Practical Common Ruby - Do you really need macros?

November 26, 2007

Chapter 9 of Practical Common Lisp has us diving deeper into Lisp macros by creating a test framework, which I will of course, try to port to Ruby. The more I continue to work on this, the more I’m convinced this is a great way to learn a new language.

First off, you are forced to digest the examples that you convert, because you have to think about what the code is doing and how you would do it in the language that you know. The bonus payoff is that you now have code in both languages to compare, so you can evaluate the two languages side-by-side. So on with the code.

Right off bat we run into a small problem, which is that you can’t have “+” or “*” characters in the name of a method in Ruby. That’s ok, we can work around that:

def test_plus
  1 + 2 == 3 &&
  1 + 2 + 3 == 6 &&
  -1 + -3 == -4
end

Which works:

paulbarry@paulbarry: ~/projects/practical_common_ruby $ irb
irb(main):001:0> load 'chap9.rb'
=> true
irb(main):002:0> test_plus
=> true

Next we tackle the version with test case info:

def test_plus
  puts "#{eval("1 + 2 == 3") ? 'pass' : 'FAIL'} ... 1 + 2 == 3"
  puts "#{eval("1 + 2 + 3 == 6") ? 'pass' : 'FAIL'} ... 1 + 2 + 3 == 6"
  puts "#{eval("-1 + -3 == -4") ? 'pass' : 'FAIL'} ... -1 + -3 == -4"
end

Now a little refactoring by adding report_result:

def report_result(result, form)
  puts "#{result ? 'pass' : 'FAIL'} ... #{form}"
end

def test_plus
  report_result(1 + 2 == 3, "1 + 2 == 3")
  report_result(1 + 2 + 3 == 6, "1 + 2 + 3 == 6")
  report_result(-1 + -3 == -4, "-1 + -3 == -4")
end

This isn’t really very clean, first of all because of the obvious duplication of the actual code and the name, but secondly, we have our code as an argument to the report_result method. This works in lisp, because everything is just an expression, but in Ruby, once this method gets more complicated, this is going to be ugly. So I’m going to refactor report_result to allow for two conditions. The first just takes the code as a string and evals it, using the code as the “name” of the test. Second, the string is the “name” and the block has the code. I think this makes sense, because even in lisp, if the code is some long, multiline expression, you aren’t going to want that to be the name of the test, you are going to want to supply a shorter, more descriptive name.

def report_result(test)
  if block_given?
    result = yield
  else  
    result = eval(test)
  end
  puts "#{result ? 'pass' : 'FAIL'} ... #{test}"
end

def test_plus
  report_result "1 + 2 == 3"
  report_result "1 + 2 + 3 == 6"
  report_result "adding negative numbers" do
    -1 + -3 == -4
  end
end

The next refactoring is to define a check method that takes multiple “tests” and prints the results of each one, mainly because the repeated calls to report_result is considered unappealing. I’m not sure there is a particularly cleaner version of this in Ruby, but here’s the best I’ve got:

def report_result(test)
  if block_given?
    result = yield
  else  
    result = eval(test)
  end
  puts "#{result ? 'pass' : 'FAIL'} ... #{test}"
end

def check(*tests)
  tests.each do |t|
    if t.is_a?(Hash)
      report_result t.keys.first, &t.values.first
    else
      report_result t
    end
  end
end

def test_plus
  check(
    "1 + 2 == 3",
    "1 + 2 + 3 == 6",
    "adding negative numbers" => lambda{ -1 + -3 == -4 }
  )
end

I found this tutorial on Ruby’s Procs and Blocks to be helpful while working on this. In reality, I think most Ruby programmers would stick to the original version of test_plus over this version that uses Hash and Lambdas, but I’ll stick with this one because it’s closer to the actual code in the example. A couple of small changes give us the ability to track if the test has a failure:

def report_result(test)
  if block_given?
    result = yield
  else  
    result = eval(test)
  end
  puts "#{result ? 'pass' : 'FAIL'} ... #{test}"
  result
end

module Enumerable
  def each?
    result = true
    each do |i|
      result = false unless yield(i)
    end
    result
  end
end

def check(*tests)
  tests.each? do |t|
    if t.is_a?(Hash)
      report_result(t.keys.first, &t.values.first)
    else
      report_result(t)
    end
  end
end

I choose to implement the combine_results function from the example as an iterator called each? mixed-in to Enumerable, so that we can call it in the idiomatic Ruby way tests.each?. each? is similar to the all? method that already exists in Enumerable, except that all? short-circuits and stops iterating through over the items once the block evaluates to false. We want to always perform the block on each item in the enumerable, and then return false if any are false. I choose to name it each? rather than combine_results because it’s more descriptive, more Rubyish. I’m actually kind of surprised Ruby doesn’t have a method like that.

Now this next bit require dynamic variables. This is a new concept for me and I have to say, it is pretty cool. The bad news? Ruby doesn’t have dynamic variables :(. The good news? We can fake it. Just download this code and we are ready to go. We add in a line to define the variable:

Dynamic.variable :test_name

Then we add it into our test method. We also have to do a little finagling to get them to still return the right value:

def test_plus
  result = false
  Dynamic.let :test_name => 'test_plus' do
    result = check(
      "1 + 2 == 3",
      "1 + 2 + 33 == 6",
      "adding negative numbers" => lambda{ -1 + -3 == -4 }
    )
  end
  result
end

def test_times
  result = false
  Dynamic.let :test_name => 'test_times' do
    result = check(
      "2 * 2 == 4",
      "3 * 5 == 15"
    )
  end
  result
end

Lastly we just add the dynamic variable to the print statement in report_result:

def report_result(test)
  if block_given?
    result = yield
  else  
    result = eval(test)
  end
  puts "#{result ? 'pass' : 'FAIL'} ... #{Dynamic.test_name}: #{test}"
  result
end    

So now we want to clean up this redundant test code, but alas, Ruby does not have macros. Here we go again, coming pretty close. Define a method we will use to define tests:

def test(name)
  test_name = "test_#{name}"
  method = lambda do
    result = false
    Dynamic.let :test_name => "#{Dynamic.test_name} #{test_name}" do
      result = yield
    end
    result
  end    
  Object.send(:define_method, test_name, method)
end

I’m not even going to try to explain how this code works. This is pretty dense. I’m not sure if it is more or less dense than the Lisp code. Now that we have our Ruby “macro” created, we can define our tests like this:

test "plus" do
  check(
    "1 + 2 == 3",
    "1 + 2 + 3 == 6",
    "adding negative numbers" => lambda{ -1 + -3 == -4 }
  )
end

test "times" do 
  check(
    "2 * 2 == 4",
    "3 * 5 == 15"
  )
end

test "arithmetic" do
  %w{test_plus test_times}.each? do |t|
    send t
  end
end

test "math" do
  test_arithmetic
end      

And voila, we have a feature complete version of the test framework in Ruby:

irb(main):096:0> test_math
pass ...  test_math test_arithmetic test_plus: 1 + 2 == 3
pass ...  test_math test_arithmetic test_plus: 1 + 2 + 3 == 6
pass ...  test_math test_arithmetic test_plus: adding negative numbers
pass ...  test_math test_arithmetic test_times: 2 * 2 == 4
pass ...  test_math test_arithmetic test_times: 3 * 5 == 15
=> true
irb(main):097:0> test_arithmetic
pass ...  test_arithmetic test_plus: 1 + 2 == 3
pass ...  test_arithmetic test_plus: 1 + 2 + 3 == 6
pass ...  test_arithmetic test_plus: adding negative numbers
pass ...  test_arithmetic test_times: 2 * 2 == 4
pass ...  test_arithmetic test_times: 3 * 5 == 15
=> true
irb(main):098:0> test_plus
pass ...  test_plus: 1 + 2 == 3
pass ...  test_plus: 1 + 2 + 3 == 6
pass ...  test_plus: adding negative numbers
=> true

So in conclusion, macros are one feature of Lisp that have no direct equivalent in Ruby, but you can define methods that define methods, which is pretty close to what macros do. Given Ruby’s metaprogramming features like eval, send, and define_method, etc., Do you really need macros?. The search for the answer continues…

Posted in Technology | Topics Macros, Ruby, Lisp

Practical Common Ruby

November 25, 2007

As part of my latest effort to learn Lisp, I’m going through Practical Common Lisp. After reading chapter 3, I thought it would be a cool idea to translate that chapter into a language I’m familiar with, Ruby. I’m assuming it will help point out some of the powerful aspects of Lisp, showing how accomplishing the same thing in Ruby is more difficult. So pull up Chapter 3 and follow along at home.

In order to follow along, what I’m doing is using irb, Ruby’s version of the Lisp REPL, and saving the code into a script. So create a file called chap3.rb or whatever, and then fire up irb in that same directory:

paulbarry@paulbarry: ~/projects/practical_common_ruby $ irb
irb(main):001:0> load 'chap3.rb'
=> true

Now your script is loaded and you can call whatever methods you have defined. Also, you can run load 'chap3.rb' again and it will reload the script.

So right of the bat, one different we run into is the lack of a property list in Ruby. So in Lisp you have this:

CL-USER> (getf (list :a 1 :b :c 3) :a)
1

It has properties of both a Ruby Array and a Ruby Hash. It’s just a list of items, but it’s like a Hash in that you can lookup a value by it’s key, using the getf function as shown above. So in Ruby, we could use a Hash:

irb(main):001:0> {:a => 1, :b => 2, :c => 3}[:a]
=> 1

Which works for the lookup part, but doesn’t preserve insertion order. If we need insertion order, we could use an Array of Arrays and write our own getf function:

def getf(plist, key)
  plist.each do |k, v|
    return v if key == k
  end
  nil
end

irb(main):007:0> getf([[:a, 1], [:b, 2], [:c, 3]], :a)
=> 1
irb(main):008:0> getf([[:a, 1], [:b, 2], [:c, 3]], :d)
=> nil
irb(main):009:0> getf([[:a, 1], [:b, 2], [:c, 3]], :b)
=> 2

In this case, I don’t think insertion order is necessary, so we’ll just use an Array of Hashes, as it’s a little more a part of Ruby.

Now that that’s out of the way, we’ll set up the global database and the function to make cds:

$db = []

def make_cd(title, artist, rating, ripped)
  {:title => title, :artist => artist, :rating => rating, :ripped => ripped}
end

def add_record(cd)
  $db << cd
end

As you can see, this is just a straight port of the code, which I’ll try to stick to throughout. Next up is the dump-db function. Now one little trick that Lisp’s format function has is to be able iterate through a list within the format string. This I can say Ruby doesn’t have, and it’s probably due to the fact that lists are so much a part of Lisp. Here’s a more simple example of how it works:

CL-USER> (format t "~{~a~%~}" '(1 2 3))
1
2 
3

Each element of the format string is preceded by a ~ character. So it starts with ~{, which indicates we’re working with an element that is a list. The ~} at end just closes the list of things we are doing to each item. The ~a element just effectively prints the argument and the ~% prints a new line character.

As the author points out, this isn’t all that different from the ruby % format operator, except for the functionality to iterate over a list. So we’ll have to do that ourselves in the Ruby version, but luckily it’s not that much work. Here’s our first take at it:

def dump_db
  $db.each do |cd|
    cd.each do |k,v|
      puts "%-10s%s" % ["#{k.to_s.upcase}:", v]
    end
  end
end

Here’s what that results in:

paulbarry@paulbarry: ~/projects/practical_common_ruby $ irb
irb(main):001:0> load 'chap3.rb'
=> true
irb(main):002:0> add_record(make_cd('Largo','Brad Mehldau',9,true))
=> [{:artist=>"Brad Mehldau", :rating=>9, :ripped=>true, :title=>"Largo"}]
irb(main):003:0> add_record(make_cd('Junta','Phish',9,true))
=> [{:artist=>"Brad Mehldau", :rating=>9, :ripped=>true, :title=>"Largo"}, 
    {:artist=>"Phish", :rating=>9, :ripped=>true, :title=>"Junta"}]
irb(main):004:0> dump_db
ARTIST:   Brad Mehldau
RATING:   9
RIPPED:   true
TITLE:    Largo

ARTIST:   Phish
RATING:   9
RIPPED:   true
TITLE:    Junta

This works except for one issue, the items print out an arbitrary order, since Ruby’s Hash doesn’t preserve insertion order :(. So we can specify the order in the function if that’s something we care about:

def dump_db
  $db.each do |cd|
    %w{title artist rating ripped}.each do |f|
      puts "%-10s%s" % ["#{f.upcase}:", cd[f.to_sym]]
    end
    print "\n"
  end
end

But for the rest of the example I’m going to stick with the random ordered version. The reason I like that it that it prints out everything in the record, even if we add new fields.

So I’m getting tired of re-adding the data, so I’m going to jump down to saving and loading the data.

def save_db(filename)
  open(filename, 'w') do |file|
    file.puts $db.inspect
  end
end

def load_db(filename)
  $db = eval(open(filename){|f| f.read})
end

Here it is in action:

paulbarry@paulbarry: ~/projects/practical_common_ruby $ irb
irb(main):001:0> load 'chap3.rb'
=> true
irb(main):002:0> add_record(make_cd('Largo','Brad Mehldau',9,true))
=> [{:artist=>"Brad Mehldau", :rating=>9, :ripped=>true, :title=>"Largo"}]
irb(main):003:0> add_record(make_cd('Junta','Phish',9,true))
=> [{:artist=>"Brad Mehldau", :rating=>9, :ripped=>true, :title=>"Largo"}, 
    {:artist=>"Phish", :rating=>9, :ripped=>true, :title=>"Junta"}]
irb(main):004:0> save_db "my-cds.db"
=> nil
irb(main):005:0> quit
paulbarry@paulbarry: ~/projects/practical_common_ruby $ irb
irb(main):001:0> load 'chap3.rb'
=> true
irb(main):002:0> load_db "my-cds.db"
=> [{:artist=>"Brad Mehldau", :rating=>9, :ripped=>true, :title=>"Largo"}, 
    {:artist=>"Phish", :rating=>9, :ripped=>true, :title=>"Junta"}]
irb(main):003:0> dump_db
ARTIST:   Brad Mehldau
RATING:   9
RIPPED:   true
TITLE:    Largo

ARTIST:   Phish
RATING:   9
RIPPED:   true
TITLE:    Junta

So now we jump back up to “Improving the User Interaction”. This basically translates directly into Ruby, except that we have to create our own y_or_n_p, which is trivial:

def prompt_read(prompt)
  print "#{prompt}: "
  gets.chomp
end

def y_or_n_p(prompt)
  case prompt_read(prompt+" [y/n]").upcase
  when 'Y': true
  when 'N': false
  else y_or_n_p(prompt)
  end
end

def prompt_for_cd
  make_cd(
    prompt_read("Title"),
    prompt_read("Artist"),
    prompt_read("Rating").to_i,
    y_or_n_p("Ripped")
  )
end

def add_cds
  loop do 
    add_record prompt_for_cd
    break unless y_or_n_p("Another?")
  end
end

And here it is in action:

irb(main):021:0> add_cds
Title: Wormwood
Artist: moe.
Rating: 10
Ripped [y/n]: y
Another? [y/n]: y
Title: Animals
Artist: Pink Floyd
Rating: 8
Ripped [y/n]: n
Another? [y/n]: n
=> nil
irb(main):022:0> dump_db
ARTIST:   Brad Mehldau
RATING:   9
RIPPED:   true
TITLE:    Largo

ARTIST:   Phish
RATING:   9
RIPPED:   true
TITLE:    Junta

ARTIST:   moe.
RATING:   10
RIPPED:   true
TITLE:    Wormwood

ARTIST:   Pink Floyd
RATING:   8
RIPPED:   false
TITLE:    Animals

So on to querying the database. We’ll use Ruby’s find_all iterator rather than Lisp’s remove-if-not function. I’ll go through each function as the chapter does. First up, selecting an artist:

def select_by_artist(artist)
  $db.find_all{|cd| cd[:artist] == artist}
end

So then we refactor that to pass the selector function (Proc, in Ruby terminology) into the method, and we have a method to create the selector:

def select(selector)
  $db.find_all{|cd| selector.call(cd) }
end

def artist_selector(artist)
  lambda{|cd| cd[:artist] == artist}
end

We call this in Ruby like this:

irb(main):034:0> select artist_selector(“Phish”)

So next we build the “where” selector, and we’ll use Ruby’s Hash to stand in for keyword parameters, which are very similar:

def where(p={})
  lambda do |cd|
    (p.has_key?(:title) ? cd[:title] == p[:title] : true) &&
    (p.has_key?(:artist) ? cd[:artist] == p[:artist] : true) &&
    (p.has_key?(:rating) ? cd[:rating] == p[:rating] : true) &&
    (p.has_key?(:ripped) ? cd[:ripped] == p[:ripped] : true)
  end
end

We can call it like this to verify that it works:

irb(main):066:0> select where(:artist => "moe.", :rating => 10)
=> [{:artist=>"moe.", :rating=>10, :ripped=>true, :title=>"Wormwood"}]
irb(main):067:0> select where(:artist => "moe.", :rating => 9)
=> []  

By the way, jumping ahead a bit, there’s no need to explicitly list each field in the where function. We can simplify that down like this:

def where(p={})
  lambda do |cd|
    r = true
    p.each do |k,v|
      unless cd[k] == v
        r = false
        break
      end
    end
    r
  end
end

So now if you add fields to the cd record, you don’t have to touch any of these methods. Onto the update method. This time we’ll have Ruby’s each iterator stand in for Lisp’s mapcar. Also, for the sake of simplicity, we’ll modify the actual record in the database, instead of making a copy of the database and updating the global variable to point to the new database.

def update(selector, values={})
  $db.each do |row|
    if selector.call(row)
      values.each do |k,v|
        row[k] = v
      end
    end
  end
end

Again, I feel this is considerably more readable than the Lisp version, plus it doesn’t require explicitly listing each field. I suppose readability is in the eye of the beholder. I would imagine Lisp programmers find the end statements in the method above as annoying as Lisp new-comers find the parenthesis in Lisp code, but as you become familiar with the language and the syntax, those annoyances just fade away.

Here it is in action, after adding one more Phish album to the collection, which is never a bad thing:

irb(main):014:0> select where(:artist => "Phish")
=> [{:title=>"Junta", :artist=>"Phish", :rating=>9, :ripped=>true}]
irb(main):015:0> add_cds
Title: Lawnboy
Artist: Phish
Rating: 8
Ripped [y/n]: y
Another? [y/n]: n
=> nil
irb(main):016:0> save_db "my-cds.db"
=> nil
irb(main):017:0> select where(:artist => "Phish")
=> [{:title=>"Junta", :artist=>"Phish", :rating=>9, :ripped=>true}, 
    {:title=>"Lawnboy", :artist=>"Phish", :rating=>8, :ripped=>true}]
irb(main):018:0> update where(:artist => "Phish"), :rating => 7
=> [{:title=>"Largo", :artist=>"Brad Mehldau", :rating=>9, :ripped=>true}, 
    {:title=>"Junta", :artist=>"Phish", :rating=>7, :ripped=>true}, 
    {:title=>"Wormwood", :artist=>"moe.", :rating=>10, :ripped=>true}, 
    {:title=>"Animals", :artist=>"Pink Floyd", :rating=>8, :ripped=>false}, 
    {:title=>"Lawnboy", :artist=>"Phish", :rating=>7, :ripped=>true}]
irb(main):019:0> select where(:rating => 7)
=> [{:title=>"Junta", :artist=>"Phish", :rating=>7, :ripped=>true}, 
    {:title=>"Lawnboy", :artist=>"Phish", :rating=>7, :ripped=>true}]

And for sake of completedness, here’s the delete:

def delete(selector)
  $db.delete_if{|cd| selector.call(cd) }
end

And that gives us the whole thing weighing in at 88 lines. It’s longer than the Lisp version in terms of number of lines simply because the end statements sit on their own line. We also don’t have the duplication that is removed in the final section of this chapter, so that’s not necessary.

But the essence of the final section is macros, which seems to be one of the most unique and powerful features of Lisp. In this particular chapter, we’ve managed to write code that is just as powerful and maintainable, and possibly more readable, without macros. But as I get deeper into Lisp, I’m sure I’ll find examples where that’s not the case. One observation I have from this so far is that I’ve never used Ruby’s clearly Lisp-inspired lambda feature in my day-to-day Rails work, but maybe I should be.

Posted in Technology | Topics Ruby, Lisp

LISP - My Personal NBL

November 23, 2007

Practical Common Ruby

In the not to distant past I considered myself a “Java” programmer. For my job, I programmed in Java, although I knew and used some other languages to varying degrees, including Perl, PHP, Python and JavaScript. I started to learn Ruby due to all the hype Rails was getting. I really liked Ruby and now program in Ruby almost exclusively, so I suppose I can drop the “Java” from my informal title. I don’t think I’ll ever call myself a “Ruby” programmer because one of the many things I have learned while learning Ruby is that learning different languages makes you a better programmer in general. You learn new techniques that you can apply to programming in almost any language.

So throughout my career I expect to continue learn other languages. A few that are on my horizon for now are Smalltalk, C/C++, Haskell, Erlang and maybe dive a bit deeper into Python, but for now those are all on the back burner. A few years ago I was programming in Java in my day job and learning Ruby in my spare time. Now I’m programming in Ruby in my day job and learning Lisp in my spare time.

There are several reason for wanting to learn Lisp, but I would say Paul Graham is definitely at the top of that list. I read his book Hackers and Painters and I suggest you do as well. The book is actually a collection of essays, most of which you can read online:

  1. Why Nerds Are Unpopular
  2. Hackers and Painter
  3. What You Can’t Say
  4. Good Bad Attitude
  5. The Other Road Ahead
  6. How to Make Wealth
  7. Mind the Gap
  8. A Plan for Spam
  9. Taste for Makers
  10. Programming Languages Explained
  11. The Hundred-Year Language
  12. Beating the Averages
  13. Revenge of the Nerds
  14. The Dream Language
  15. Design and Research

It looks like there are some good free online resources for learning Lisp. I’m gonna start with Practical Common Lisp, and I’ve also been going through Structure and Interpretation of Computer Programs. It is a computer programming course that is available online. The book is here and videos of the lectures are here.

Posted in Technology | Topics Lisp