Installing DBSlayer on Mac OS X Snow Leopard

February 2, 2010

DBSlayer is a tool that will wrap your MySQL database with an HTTP REST/JSON API. Here's how to get it installed on Snow Leopard using Macports. First, make sure you have all the dependencies install via Macports:

$ sudo port install db46
$ sudo port install apr
$ sudo port install apr-util
$ sudo port install mysql5-server

Then, if you try to download the source and install it:

$ cd ~/src
$ wget http://code.nytimes.com/downloads/dbslayer-beta-12.tgz
$ tar xzf
$ cd dbslayer
$ ./configure
$ make
$ make install

You'll run into this error:

ld: library not found for -ldb-4.6
collect2: ld returned 1 exit status
make[1]: *** [dbslayer] Error 1
make: *** [all-recursive] Error 1

Instead, pass these options to configure:

$ ./configure CFLAGS="-L/opt/local/lib/db46" \
--with-apr-1-config=/opt/local/bin/apr-1-config \
--with-apu-1-config=/opt/local/bin/apu-1-config \
--with-mysql-config=/opt/local/lib/mysql5/bin/mysql_config

Now try to install again:

$ make clean
$ make
$ sudo make install

Next, create the config file, which at it's most basic, should look something like this:

[my_db]
database=my_db
host=localhost
user=root

Now when you start dbslayer, make sure to give it the full path to the config file:

$ dbslayer -c ~/stuff/my_db.cnf -s my_db

dbslayer starts in the background, so to restart it, you have to find the process and kill it manually. It also doesn't say whether it worked or not, it just backgrounds with no output. Here's a little Ruby script to try it out:

require 'open-uri'
require 'rubygems'
require 'json'
require 'pp'

def query_url(sql)
  query_hash = { "SQL" => sql }
  url_args = URI.encode(query_hash.to_json)
  "http://localhost:9090/db?#{url_args}"
end

def exec_query(sql)
  url = query_url(sql)
  open(url) do |f|
    yield JSON.parse(f.read)
  end
end

exec_query "select * from stuff limit 1" do |res|
  pp res
end

Posted in Technology | Tags MacOSX, MySQL, REST, SnowLeopard, DBSlayer | 6 Comments

Node.js talk at Baltimore/JavaScript Users Group

February 1, 2010

Just a quick heads up that I'll be talking about Node.js at this Wednesday's Baltimore/JavaScript Users Group meeting. This will be an introductory talk but will quickly dive into some code examples with Q & A. The talk, just as node.js itself, centers around using Evented, Asynchronous I/O to get better scalability out of network servers, so it should be of interest to anyone doing server-side programming in any language, Java, Perl, Ruby, Python, etc. See you there!

Posted in Technology | Tags Javascript, node.js | 0 Comments

Customizing Generators in Rails 3

January 13, 2010

As you probably already know, Rails 3 is just around the corner. There are some pretty nice features being added and one of them is the ability to customize the way the generators work. I personally prefer Haml over ERB, RSpec over Test::Unit and Factory Girl over Fixtures. So let's see how we can configure a Rails 3 app to do that.

First, follow Yehuda's instructions on how to create a Rails 3 app. Next, you have to tell Rails that you want to use Haml, RSpec and Factory Girl. First, add this somewhere in the Gemfile:

gem "haml"

only :test do
  gem "rspec"
  gem "rspec-rails"
  gem "factory_girl"
end

Then, re-run the bundler and initialize the Haml plugin:

$ gem bundle
$ bin/haml --rails .

Finally, you have to install the custom generators for the frameworks that you want to use. I found a repo on github that already had RSpec, so I forked it and added Haml and Factory Girl. Clone the repo into the lib/generators directory of your app:

$ git clone git://github.com/pjb3/rails3-generators.git lib/generators

Now, in the config/application.rb file in your app, near the bottom there is a section related to config.generators. Put this in that section:

config.generators do |g|
  g.template_engine :haml
  g.test_framework :rspec, :fixture => true, :views => false
  g.fixture_replacement :factory_girl, :dir => "spec/factories"
end

Here is where we reap the benefits of the modularity in Rails 3. What this says is that we want to use Haml as the template enging, we want to use RSpec as the test framework and we want to generate fixtures with our generated specs, but we don't want to generate view specs and that instead of fixtures, we actually want to use factory girl and we want the factories to be put into spec/factories. Whew! So does this all work?

$ script/generate rspec:install
$ script/generate scaffold person name:string dob:date age:integer male:boolean bio:text

At this point you should see that Rails has generated what we want, which is scaffolding that uses Haml for the views, RSpec for the tests and Factory Girl for the fixtures. Run the migrations, start the server and open the browser http://localhost:3000/people to see your scaffolding in action.

Now if you try to actually run rake spec, you'll get an error, at least I do. I'm not sure that RSpec 1.X is going to ever work with Rails 3. I think the intent is for RSpec 2 to be compatible with Rails 3, so keep an eye out for that.

Posted in Technology | Tags FactoryGirl, RSpec, Rails, HAML | 2 Comments

Guarding Logger Statements In Ruby

December 9, 2009

Whether you are a Java or a Ruby programmer, I'm sure you are familiar with this idiom:

require 'logger'

log = Logger.new(STDOUT)
log.level = Logger::INFO

log.debug("hello")
log.info("Done")

That's a simple logger where the log level is set to info, so the debug statement isn't logged, but the info statement is. One gotcha to look out for is something like this:

require 'logger'

log = Logger.new(STDOUT)
log.level = Logger::INFO

def fib(n)
  if n < 1
    0
  elsif n < 2
    1
  else
    fib(n-1) + fib(n-2)
  end
end

log.debug("fib(30) => #{fib(30)}")
log.info("Done")

This also just logs "Done", but it take more than a few seconds to do so. The reason why is that even though you aren't logging the string that gets passed to debug, the ruby interpreter still has to incur the cost of generating the string and passing it to debug, where it gets ignored.

If you are an old Java programmer like me, you'll probably know you can fix it like this:

require 'logger'

log = Logger.new(STDOUT)
log.level = Logger::INFO

def fib(n)
  if n < 1
    0
  elsif n < 2
    1
  else
    fib(n-1) + fib(n-2)
  end
end

if log.debug?
  log.debug("fib(30) => #{fib(30)}")
end
log.info("Done")

That works, but it's not the Ruby way of doing it. It's the idiomatic way of doing it in Java, but that is due to the fact that Java doesn't have anonymous functions nor a concise syntax for creating them. The Ruby way of doing it is:

require 'logger'

log = Logger.new(STDOUT)
log.level = Logger::INFO

def fib(n)
  if n < 1
    0
  elsif n < 2
    1
  else
    fib(n-1) + fib(n-2)
  end
end

log.debug { "fib(30) => #{fib(30)}" }
log.info("Done")

The difference between this version and the original is that instead of passing a string to debug, we pass a block that returns a string when it is called. We don't have to wrap it in an if statement because the block can be conditionally evaluated based on the current log level.

The difference between the if statement and the block is admittedly minor. That being said, prefer the block syntax. :)

The important thing to remember is that if you have a debug statement that does any kind of calculating, pass it a block instead of just a string to avoid the overhead associated with unnecessarily building the string.

Posted in Technology | Tags RubyOnRails, Ruby, Java | 2 Comments

Performance Testing with Httperf

November 10, 2009

If you need to do some performance testing of your web app, one tool that is pretty easy to use is httperf. I recommend watching the Peepcode screencast on httperf to get some good tips on how to doing performance testing. There is also some reading material on httperf created by Ted Bullock available. These are great resources, but here's the quick and dirty on how to get httperf going.

First, to install httperf, if you are using Macports, you can simply do sudo port install httperf, which is what I did. Alternatively, you can get httperf from the httperf download page.

Once you've got it installed, make sure the site you are testing on is running with all the production settings turned on. The Peepcode screencast goes into detail about how to do that for a Rails app. Then, if you are trying to test the /profile page on myfancysocialnetwork.com, you would run a command like this:

httperf --server myfancysocialnetwork.com --port 80 --uri /profile --num-conns 100

This will hit the profile page 100 times and report back with statistics like the minimum response time, maximum response time and the average response time, including standard deviation. You will certainly want to adjust the --num-conns option based on the page you are testing. If it's a pretty slow page with response times over a second or two, then 100 is probably ok. If it's a more well behaved page that renders in a few hundred milliseconds, then a large number like 10000 might be better. The goal is to make sure that it runs long enough to generate enough samples to make the results statistically relevant. You'll also want to read the documentation/watch the screencast to get info on other options like rate to create different kinds of test scenarios.

Posted in Technology | Tags Httperf | 2 Comments

<< Newer Articles   Older Articles >>