Installing DBSlayer on Mac OS X Snow Leopard

February 2, 2010

DBSlayer is a tool that will wrap your MySQL database with an HTTP REST/JSON API. Here's how to get it installed on Snow Leopard using Macports. First, make sure you have all the dependencies install via Macports:

$ sudo port install db46
$ sudo port install apr
$ sudo port install apr-util
$ sudo port install mysql5-server

Then, if you try to download the source and install it:

$ cd ~/src
$ wget http://code.nytimes.com/downloads/dbslayer-beta-12.tgz
$ tar xzf
$ cd dbslayer
$ ./configure
$ make
$ make install

You'll run into this error:

ld: library not found for -ldb-4.6
collect2: ld returned 1 exit status
make[1]: *** [dbslayer] Error 1
make: *** [all-recursive] Error 1

Instead, pass these options to configure:

$ ./configure CFLAGS="-L/opt/local/lib/db46" \
--with-apr-1-config=/opt/local/bin/apr-1-config \
--with-apu-1-config=/opt/local/bin/apu-1-config \
--with-mysql-config=/opt/local/lib/mysql5/bin/mysql_config

Now try to install again:

$ make clean
$ make
$ sudo make install

Next, create the config file, which at it's most basic, should look something like this:

[my_db]
database=my_db
host=localhost
user=root

Now when you start dbslayer, make sure to give it the full path to the config file:

$ dbslayer -c ~/stuff/my_db.cnf -s my_db

dbslayer starts in the background, so to restart it, you have to find the process and kill it manually. It also doesn't say whether it worked or not, it just backgrounds with no output. Here's a little Ruby script to try it out:

require 'open-uri'
require 'rubygems'
require 'json'
require 'pp'

def query_url(sql)
  query_hash = { "SQL" => sql }
  url_args = URI.encode(query_hash.to_json)
  "http://localhost:9090/db?#{url_args}"
end

def exec_query(sql)
  url = query_url(sql)
  open(url) do |f|
    yield JSON.parse(f.read)
  end
end

exec_query "select * from stuff limit 1" do |res|
  pp res
end

Posted in Technology | Tags MacOSX, MySQL, REST, SnowLeopard, DBSlayer | 6 Comments

Ruby on Struts

September 6, 2008

Once upon a time, there was a web MVC framework called Struts. Struts was one of the original catalysts of web frameworks based on the MVC pattern, but it was written in Java and required copious amounts of XML to configure your application. One of the many things defined in the XML were the "Action Mappings". An Action Mapping essentially mapped a specific URL pattern to a specific Java class that would be responsible for handling that request.

Then came Ruby on Rails, which eliminated the need for these XML configuration files by using Convention Over Configuration. The way this works in Rails is that if a request is sent to the url /users/new, Rails will call the new method of the UsersController class to handle the request. Rails has a feature called routing that is used to map unconventional url patterns to specific controller actions. Then came RESTful Rails and the convention over configuration was gone.

With RESTful Rails, the request path simply represents what you want to perform an operation on, the Resource, and the HTTP method specifies what you want to do. This all makes sense, but one problem is that RESTful urls do not conform to the url convention. This means every action must be defined in the routing. A shortcut was added to the routing to allow one expression to define the 7 typical methods the controller for a resource will have, but any additional actions must be explicitly defined.

I've been recently working with an experienced Java developer with some familiarity Rails. He was confused by the way RESTful routing works and what paths the named route methods would generate. When he asked what benefits all this provides over the original /controller/action/id pattern, where only non-standard routes had to be mapped, I struggled to find any. I realized that I was doing this just because it was now the "Rails Way". He said this reminds him of Struts, and after some arguing and thinking about it, I realized he was right.

So it seemed unbelievably coincidental that we both found ourselves together today in a talk titled Unconvental Wisdom by Bruce Tate. In today's talk, Bruce pointed out that RESTful Rails adds complexity to Rails, which makes it harder to explain how Rails works to newcomers to Rails. I'm very interested to see how this talk is received by the Rails community, so if you weren't at the talk today, look for it on Confreaks in the next few weeks.

So looking back on RESTful Rails applications that I've developed, if they reach even a level of medium complexity, you end up with 50 lines or so in the routes.rb, with at least a handful of nested and custom routes, in addition to all of the resources. Imagine this as the admin interface for a simple blog:

map.namespace(:admin) do |admin| 
  admin.resources :articles, :has_many => :comments, :member => { :publish => :post }
  admin.resources :categories, :has_many => :articles
  admin.resources :comments, :belongs_to => :article
  admin.resources :tags, :has_many => :articles
end

If you are an experienced Rails developer, you can decipher this right away. But is this really easy understand? Or wouldn't convention over configuration be easier:

map.connect "/admin/:controller/:action/:id"

And what about named routes like new_admin_article_comment_path(@article)? Is that really more clear than admin_path("/comments/new", :article_id => @article)"? Do you really care if the urls are /comments/new?article_id=1 or /articles/1/comments? Maybe I missing something, but I'm starting to like the sound of idea of having almost nothing in my routes and not calling dynamically generated methods everywhere to just to build simple paths. After all, no code is faster than no code.

Posted in Technology | Tags Ruby, Rails, REST | 28 Comments

Separating HTML from the REST with RADAR

March 30, 2007

Dave Thomas has posted a really good article entitled The RADAR Architecture: RESTful Application, Dumb-Ass Recipient. I can already see the RADAR books published by O'Reilly, Manning, Wrox, Apress and of course, The Pragmatic Programmers, on the shelves at Barnes & Noble. The concept is inspired by RESTful Rails, but proposes an alternate approach from the one currently used by Rails.

If you aren't familiar with RESTful Application Architecture, I'll try to give a brief crash course. The basic idea is that you treat everything in your application as a resource. Each resource has a few different URLs you use to view and edit the data. You use the HTTP methods has they were originally intended to indicate to the action you wish to take on the data.

According to the [Wikipedia article on HTTP], version 1.0 of the spec was released in May 1996, almost 11 years ago. For years, the developers of web browsers, and therefore by association, the developers of web applications, have completed ignore most of the HTTP methods that were included in the spec. We all know about GET and POST, GET is used to retrieve a document, and POST is used to send data to the server. But there were other methods that were in the spec, specifically PUT and DELETE. Sadly, some browsers (I don't know which ones do an don't) don't support PUT and DELETE, so therefore they cannot be used by HTML-browser-based web applications. But let's ignore HTML web browsers and just look at the spec for a minute.

(As a sidebar, somewhere there is someone who worked on the original HTTP spec screaming "it's 'bout time!". It's funny how sometimes in the world of technology, it takes years for people to start using a particular technology how it was originally intended, and when they finally do, it makes a big impact.)

If you think about it, a typical database driven, web application, it's really just a glorified database front-end. It allows end users to perform CRUD operations on data stored in the database. As it turns out, the original HTTP methods map nicely to the CRUD operations:

Create = POST
Retrieve = GET
Update = PUT
Delete = DELETE

Most of the time, there are 2 variants of the retrieve operation. First is the list, where you see a list of all of the resources of a given type, and the second is details, where you see the details of a given resource. So, with that, you might have the following URLs associated with working with articles:

GET     /articles    => Retrieve a list of all articles
GET     /articles/1  => Retrieve the details for article "1"
POST    /articles    => Create a new article
PUT     /articles/1  => Update article "1"
DELETE  /articles/1  => Delete article "1"

So there you have it, the basics of a RESTful Application Architecture. The problem, as Dave points out, is that browsers, without the use of JavaScript, Flash, Java Applets or ActiveX plugins, are just dumb terminals. They assume the format of the data that they receive from the web server will be HTML, therefore they just render the graphical representation of the HTML they receive in response to an HTTP request. A simple HTML web browser cannot display data, and then render a form to allow the user to edit it, without making another request to the server. The browser relies on the server to generate an HTML form. So this means we have to add more responsibility to the server, and therefore we need more URLs, so the server knows which type of view to include in the HTML along with the data. So, with a Rails application, you end up with URLs like these, to support a purely HTML RESTful application:

GET     /articles         => Retrieve a list of all articles
GET     /articles/1       => Retrieve an HTML page that displays article "1"
GET     /articles/new     => Retrieve an HTML form to be used to create a new article
POST    /articles         => Create a new article
GET     /articles/1;edit  => Retrieve an HTML form to be used to update article "1"
PUT     /articles/1       => Update article "1"
DELETE  /articles/1       => Delete article "1"

But in the Web 2.0 world that we live in, we don't want our applications to be just simple HTML pages. We want to use JavaScript (specifically AJAX), Flash (Flex, Laszlo, etc.) or possibly a standalone desktop application. But, we also live in a world where we need to support JavaScript-less, Flash-less, Web 1.0 browsers, so we need to design our applications to support both. If we are using a client that has the ability to create forms and different views of data, then we can go back to the pure, simply REST architecture, using XML, JSON, etc. for the format, because the client will generate the display, we don't have to embed the HTML in the response. These types of clients are generally referred to as a Rich Internet Application, RIA for short.

Rails has a way to handle this, which is the use of the respond_to method. Jamis Buck describes how to use respond_to to support multiple client formats in this article, but the summary is that if the client asks for XML, you give it XML, if the client asks for HTML, you give it HTML.

In Dave's article, he suggests an alternative architecture. You first build a pure REST application server, that isn't clutter with the extra methods required by HTML-based applications. Therefore it only responds to the original 5 HTTP method/URI combinations that I listed above. I would imagine you would still use this in conjunction with respond_to in order to give the client the data in various formats, like XML, JSON, etc, but those would only be data formats, not formats like HTML that include visual elements. Then, you have a "proxy" server application, that receives HTTP requests from a dumb browser, retrieves the data from the REST application, then formats the data with HTML, and returns the HTML to the dumb browser. Then, when you create an RIA, it can talk directly to the REST application server.

You could implement this type of architecture without separate applications, one for the HTML-client proxy (a.k.a RADAR) and the one for the REST API. You could create a single application with separate layers to support the different client types. A common application architecture contains 3 layers, presentation, business logic, and persistence. The presentation layer could be the RADAR HTML-client proxy, and business logic could be the REST API, and the persistence layer could be your typical Data Access Object implementation, or in the case of Rails, ActiveRecord. If everything is in one server/cluster process, it actually would make more sense to have 2 different presentation layers, the RADAR layer and the REST API layer, and your business layer would be application code that each layer communicates with, that is modeled directly after the REST API.

This is just one of example of the interesting application design concepts that are coming out of the Rails community, and these are application design concepts that could be applied in any language/platform. Java developers who are turning a blind eye to Rails and application frameworks built in other languages are missing out on an opportunity to learn from these concepts.

Posted in Technology | Tags RADAR, Rails, REST | 0 Comments