Paul Barry

My Git Workflow

January 20, 2009

Inspired by Michael Ivey’s recent “My Git Workflow” article, I’ve decided to write up one of my own. Really this article should be titled “My Git SVN Workflow”, because that’s how I use it most of the time. At BrowserMedia, we use Subversion (SVN) for version control. I’m not an advanced user of Git by any stretch of the imagination, but here’s my experience from switching from SVN to Git.

First off, for small projects that I am doing on my own, I use just straight git, and back it up to my own server as I described in the Quick and Dirty Git Repository article. For stuff that I want to be publicly visible, I push it to github. But for BrowserMedia internal projects, here’s what I do.

Assuming it’s a rails project, I create the rails app and import it using SVN:

$ rails myproject -d mysql
$ cd rails
$ svn import -m "Initial Rails 2.2.2 App" https://svn.browsermedia.com/myproject/trunk
$ cd ..
$ rm -rf myproject
$ svn co https://svn.browsermedia.com/myproject/trunk myproject
$ svn remove log/*
$ svn commit -m "removing all log files from subversion"
$ svn propset svn:ignore "*.log" log/
$ svn update log/
$ svn commit -m "Ignoring all files in /log/ ending in .log"
$ svn remove tmp/*
$ svn propset svn:ignore "*" tmp/
$ svn update tmp/
$ svn commit -m "Ignoring all files in /tmp/"
$ cd ..
$ rm -rf myproject

Most of this you will recognize from the Rails Wiki article on How To Use Rails With Subversion. So now the project is setup so that anyone can use SVN to work on the project.

One of the cool things about Git is that you can use Git SVN to work with git locally and push your changes to the main SVN repository. I’ve been using this for months now with no problems. So now let’s start working on our project with git:

$ mkdir myproject
$ cd myproject
$ git svn init -s https://svn.browsermedia.com/myproject

Couple of things to note here is that first you have to make the directory for your project first, then change into the directory and run the git command from there. Also, the URL you use is the directory that contains trunk (and possibly branches and tags, if you have created those). The -s means you are using a standard SVN layout. Once you have done this, you run this command to sync up your local repository with SVN:

$ git svn fetch

In this case, this is a fresh project, so if should go pretty quick, but if you are doing this with an existing larger project, this could take quite some time, because this is not just fetching the latest version of the code, but the entire history of the repository.

To make working with git a little easier, I have some aliases defined to make things go a bit quicker:

alias gs='git status'
alias ga='git add .'
alias pull='git pull'
alias push='git push'
alias gc='git commit -m'
alias gca='git commit -a -m'
alias svnpull='git svn rebase'
alias svnpush='git svn dcommit'

And then most of the projects I work on are small teams, so I follow Michael’s small team workflow pretty closely:

$ svnpull
Hack some stuff
$ gs
look at what's changed
$ git add .
$ gca "hacked some stuff"
$ svnpush

So the first thing I usually do after git svn fetch is to add a .gitignore file to the project that looks something like this:

.DS_Store
log/*.log
tmp/**/*
db/*.sqlite3
db/schema.rb

One nice thing I’ve observed about using Git over SVN is the way it handles directories of files. Git puts one .git directory at the top level of your project, which is where it keeps the info about the repository, including the local repository itself. SVN puts a .svn directory in every directory of your project. This only has the info about the repository, which files you have checked out, etc. There is no local repository of course, the repository is on the server.

A case where this makes things easier is managing external dependencies. Let’s say you have a gem that you can’t or don’t want to install on your server. For most gems I recommend installing them on the server, but in some cases that doesn’t always work. So in that case, you do rake gems:unpack to put the gem into the vendor directory. Now let’s say you need to update that gem. Again, this is not the normal case, but if you are creating your own gems for your project, you might want to do this. Here’s what you can do with git:

$ cd important_gem
$ rake install #installs version 1.0.0 of important_gem locally
$ cd ../myproject
$ rm -rf vendor/gems/important_gem-1.0.0
$ rake gems:unpack

The case here is that myproject already has a version of important_gem-1.0.0 checked in, but you’ve modified important_gem-1.0.0, so you want to get those updates into myproject. You only do this in a situation where you haven’t released version 1.0.0 of important_gem yet.

Now if you run gs (short for git status), you will see that git has just figured out what happened. Whether files were removed, renamed, added or modifed, git knows. You can just run git add . and then gca "updated important_gem" and all is well. No need for SVN externals. With SVN, if you wanted to manage your project like that, you are kind of out of luck. The reason is that when you did rm -rf vendor/gems/important_gems-1.0.0, all of the .svn directories got destroyed. After rake gems:unpack, the files are back, but not the .svn directories, hence SVN is confused and frankly, I never did figure out what you are supposed to do about this.

There are other cases where this is useful besides working on a gem. For example, with BrowserCMS were are using FCKEditor, which comes in a zip file that you just unpack into public/fckeditor. To put in a new version of FCKEditor, you can just do:

$ cd public
$ rm -rf fckeditor
$ mkdir fckeditor
$ cd fckeditor
$ unzip ~/Downloads/fckeditor-x-y-z.zip
$ cd ../..
$ gs
$ git add .
$ gca "Upgraded to FCKeditor version x.y.z"

There are probably some errors or inefficiencies in this process and if so, let me know in the comments, but I really like working with Git SVN, which allows me to have my cake and eat it too.

Posted in Technology | Topics Git, SVN, Subversion, Rails | 2 Comments

BrowserCMS on Rails

December 31, 2008

If you haven’t heard the news already, BrowserMedia will be releasing BrowserCMS as an open source product, based on Rails! BrowserCMS 2.0 is a proprietary J2EE product and we’ve been working on BrowserCMS 3.0, a Rails-based version of the product for the last few months. Patrick Peak and I will be giving a few presentations in the next few months covering some of the motivations and lessons learned, as well as previewing some of the features of the product. First up will be the DC Ruby Users Group on Thursday, January 8th at 7:00pm. After that, we’ll be up in Baltimore for B’More on Rails on Tuesday, January 13th at 7:00pm. Then we’re headed to Orlando, Florida for Acts As Conference on Saturday, February 7th at 10:00am.

BrowserCMS 3.0 isn’t just an Open Source Rails version of BrowserCMS 2.0, it also has a new design and lots of new features. We will be finishing up the initial development phase next month and will release the initial beta version at Acts As Conference. We’ll be building out sites with BrowserCMS 3.0 for our customers at that time, so we’ll be looking for people in the community to try it out and give us feedback.

We’re excited about what we’ll be able to do by bringing together BrowserCMS and Rails. We’re also looking forward to Rails 3.0 and taking advantage of new features in Rails brought on by the Rails/Merb merger. So if you have a chance, please stop by one of the upcoming presentations, see what we’re up to and let us know what you think.

Posted in Technology | Topics BrowserCMS, Rails | 2 Comments

Named Scope: To Lambda or Not To Lambda, That Is The Question

December 18, 2008

One feature that was added to Rails in 2.1 is named scope. This is a really nice way to provide a flexible API for your controllers to query different types of data without resorting to a conditions hash or even worse, SQL. Here’s how it works. Say we have a blog with an Article model, which has a boolean field for published. We could get all published articles like this:

Article.all(:conditions => {:published => true})

But with named scope, we can do this:

Article.published.all

This is a small example so it doesn’t seem like much, but this really starts to pay of in terms of readability and encapsulation as things get more complicated. To set up the above named scope, we add this to your Article model:

named_scope :published, :conditions => {:published => true}

By doing this, we have abstracted away the implementation of what it means to be published from the controller and into the model. The advantage is that the code in the controller becomes more “intent revealing” and also if the logic for what it means to be published changes, we just change it in one place. For example, if we decide we need to keep track of when an article was published, we could change our published column from a boolean to a published_at datetime column and just change our named scope like this:

named_scope :published, :conditions => ["published_at is not null"]

If you need to further refine our query, we can chain the calls to multiple scopes together. For example, let’s say we add this to our model:

named_scope :featured, :conditions => {:featured => true}

Which allows us to do this in our controllers:

Article.published.featured.all

Now let’s say we want to create a named scope for all articles published before a given date. Ok, so we just add this to our model:

named_scope :published_before, :conditions => ["published_at < ?", ?????]

Oops. When we are defining the named scope, we don’t know what the date will be. So what do we do? That’s where lambda comes into play.

Before we do that, WTF is lambda? Lambda is a name for an anonymous function. The name comes from Lisp. The powerful thing about lambdas is that they are stored in an object and can be passed around as a variable and called at a later time. So here’s a very simple lambda:

>> double = lambda{|x| x * 2}
=> #<Proc:0x010dc4b4@(irb):1>
>> double.call 21
=> 42    

We create a lambda that takes one argument and multiplies it by two. We can also call a lambda using the [] method. So in our previous example, we could have done double[21]. I prefer call over [], because mentally the first thing I think of when I see those [] is an Array or Hash, so the call is a little more explicit as to what is going on. To each his own, but it’s good to know both ways, because you will come across both ways when reading Ruby code written by others.

This is a very powerful feature which Ruby has and many other mainstream languages lack or make it difficult to do. All functional languages that are gaining popularity now, such as Clojure, Erlang, Scala, and Haskell, also make this kind of thing easy to do.

So now that we know what lambda is, we can see how we can use it with named scope. Since we don’t know the value of the date we are trying to compare with to do our published_before named scope, we will instead give named scope a lambda that it will use at the time the SQL query is constructed:

named_scope :published_before, lambda{|date| {:conditions => ["published_at < ?", date] } }

So there we go. This is a lambda that takes the data as a parameter and returns a Hash used for the conditions of the SQL query. Now if we need last year’s featured articles, we can now do this:

Article.published_before(Time.now.beginning_of_year).featured.all
Posted in Technology | Topics Ruby, Rails | 18 Comments

Story Driven Development with Rails - Part I: Up and Running

September 16, 2008

Earlier this year at the Gotham Ruby Conference, Brian Helmkamp gave a talk on Story Driven Development. It’s a fantastic talk, I suggest that everyone view it. In it he explains how to use RSpec Users Stories combined with Webrat for full-stack executable scenario testing. He also covers what value executable scenarios provide and where executable scenarios fit into your overall web application testing strategy. This article provides you with a more detailed how-to of the steps required to get up and running with Story Driven Development with Rails.

In this article I don’t go to much into the details of how story runner works, so I would say another prerequisite for this article is Geoffrey Grosenbach’s RSpec User Stories available on PeepCode.

So let’s say we are building a rails version of this blog. First steps are the basic rails setup:

$ rails blog
$ cd blog
$ script/plugin install git://github.com/dchelimsky/rspec.git -r 1.1.4
$ script/plugin install git://github.com/dchelimsky/rspec-rails.git -r 1.1.4
$ script/generate rspec      
$ script/plugin install git://github.com/ratnikov/fixture_replacement2.git 

That gives us an rspecified Rails app, which has a stories directory, all ready for us to put our stories in. Fixture Replacement is an implementation of the factory pattern for creating test data, which is an alternative to using fixtures data in yaml files. So let’s create the first story in stories/view_articles_story.txt:

Story: View Articles

As a reader of the blog
I want to view articles
So that I can read what you have to say

  Scenario: display most recently published articles
  
    Given 5 articles have been published
    When I visit the homepage
    Then I should see the articles

A couple of things to point out. First, the naming convention I’m going with is to put the plain text story in <story_name>_story.txt and the story runner file in <story_name>_story.rb. Next I’d like to talk a little bit about this story. I choose to write this story first because it is the most important feature in the application. Generally I like to try to follow this rule. When writing a stories, always write a story for the next most important feature.

So in order to use this story, we need a runner. So let’s put this into stories/view_articles_story.rb:

require File.dirname(__FILE__) + "/helper"

run_story :view_articles

This is about concise as it gets for this file. The magic happens in stories/helper.rb. Add this to the bottom of that file:

def run_story(story_name, options={})
  with_steps_for(story_name) do
    run(
      File.join(File.dirname(__FILE__), "#{story_name}_story.txt"), 
      { :type => RailsStory }.merge(options)
    )
  end
end

What this does is allow you to call run_story to run a story. It assumes by default that you will want to use the steps that are defined in the same file with the same name as the story. We’ll probably enhance run_story in the future to do more stuff, but this is all we need for now. Now you can run either stories/all.rb or stories/view_articles_story.rb and you should get:

Running 1 scenarios

Story: View Articles

  As a reader of the blog
  I want to view articles
  So that I can read what you have to say

  Scenario: Homepage

    Given 5 articles have been published (PENDING)

    When I visit the homepage (PENDING)

    Then I should see the articles (PENDING)

1 scenarios: 0 succeeded, 0 failed, 1 pending

Pending Steps:
1) View Articles (Homepage): 5 articles have been published
2) View Articles (Homepage): I visit the homepage
3) View Articles (Homepage): I should see all 5 articles

So we now have yellow (a.k.a pending) scenarios, so step 1 is done. The next step is to write step matchers, which should make our scenario fail. So we add this to stories/view_articles_story.rb:

steps_for(:view_articles) do
  Given "$count articles have been published" do |count|
    @count = count.to_i
    @count.times {|n| create_article(:title => "Article ##{n}") }
  end
  When "I visit $path" do |path|
    get path
  end
  Then "I should see the articles" do
    @count.times do |n|
      response.should have_tag("h2", "Article ##{n}")
    end
  end
end

Make sure to add that before the run_story :view_articles line. Now you should have red (a.k.a failing) scenarios, that look something like this:

FAILURES:
    1) View Articles (Homepage) FAILED
    NoMethodError: undefined method `create_article' for   
    #<ActionController::Integration::Session:0x208bfe0>

Make sure they are failing for the right reason. For example, when I first wrote this, I forgot that all the of variables are captured as strings and I left off the call to .to_i on count. I get failing specs, but not because I haven’t implemented the code, because my steps are wrong. Remember, tests can have bugs too.

Let’s take a look at each of these steps. Inside the given step, we are calling create_article. Now as you noticed from the failure message, there is no create_article method yet. Fixture Replacement will take care of defining that for us once we have the articles model created and example_data.rb created. In the when step, we are capturing the path the user is trying to access, and then using the same rspec controller helper method we would use if this were a controller spec. get /some_path goes through the routing to call our controller with an HTTP get. Finally, in the then step, we just check that the HTML in our response has an H2 element with the title of the article in it, for each of the articles we created in the given step.

So now on to step 3, which is to write the code to make it go green. To make this easy, we’ll use the rspec scaffold generator:

$ script/generate rspec_scaffold Article title:string body:text
$ rake db:migrate
$ rake db:test:prepare

So if we run stories/view_articles_story.rb again, we get the same error. That’s because we still haven’t told Fixture Replacement about our article model. So create the file db/example_data.rb and put this in it:

module FixtureReplacement
  attributes_for :article do |a|
    a.title = "First Post!"
    a.body = "Lorem ipsum dolor sit amet, consectetuer adipiscing elit..."
  end
end

This defines the minimum set of data required to create a valid article. You can override any value by simply passing it in the hash, as we are doing in the given step. Fixture Replacement gives you two methods for each model, which are in this case, create_article and new_article, which do pretty much what you would expect. You can read more about Fixture Replacements in the README.

Last but not least, we have to include FixtureReplacement in our RSpec Story Runner, which is as simple as adding the line include FixtureReplacement somewhere near the top of stories/helper.rb, after all the other require statements.

So now if we run our story, we get something like:

1) View Articles (Homepage) FAILED
    Spec::Expectations::ExpectationNotMetError: 
    Expected at least 1 element matching "h2", found 0.
    <false> is not true.

Which is still failing, but it’s progress. Obviously the problem now is that our view isn’t outputting H2 elements for the article titles. That’s an easy fix which I’ll leave as an exercise to the reader (hint: modify the default route, edit the articles index ERB template).

One might argue that checking for H2s is checking the implementation details, which could be fragile if our designer decides to use DIVs instead, for example. That decision is up to you, personally I like to produce the correct semantic markup, have my tests validate that and apply CSS from there. If you’d rather your tests be less coupled to the DOM structure, you could just change your expectation in the then step to response.should have_text("Article ##{n}").

Hopefully that gets you up and running with Fixture Replacement and RSpec User Stories. Tune in for Part II of the series, where we cover more complex interactions with Webrat.

Ruby on Struts

September 6, 2008

Once upon a time, there was a web MVC framework called Struts. Struts was one of the original catalysts of web frameworks based on the MVC pattern, but it was written in Java and required copious amounts of XML to configure your application. One of the many things defined in the XML were the “Action Mappings”. An Action Mapping essentially mapped a specific URL pattern to a specific Java class that would be responsible for handling that request.

Then came Ruby on Rails, which eliminated the need for these XML configuration files by using Convention Over Configuration. The way this works in Rails is that if a request is sent to the url /users/new, Rails will call the new method of the UsersController class to handle the request. Rails has a feature called routing that is used to map unconventional url patterns to specific controller actions. Then came RESTful Rails and the convention over configuration was gone.

With RESTful Rails, the request path simply represents what you want to perform an operation on, the Resource, and the HTTP method specifies what you want to do. This all makes sense, but one problem is that RESTful urls do not conform to the url convention. This means every action must be defined in the routing. A shortcut was added to the routing to allow one expression to define the 7 typical methods the controller for a resource will have, but any additional actions must be explicitly defined.

I’ve been recently working with an experienced Java developer with some familiarity Rails. He was confused by the way RESTful routing works and what paths the named route methods would generate. When he asked what benefits all this provides over the original /controller/action/id pattern, where only non-standard routes had to be mapped, I struggled to find any. I realized that I was doing this just because it was now the “Rails Way”. He said this reminds him of Struts, and after some arguing and thinking about it, I realized he was right.

So it seemed unbelievably coincidental that we both found ourselves together today in a talk titled Unconvental Wisdom by Bruce Tate. In today’s talk, Bruce pointed out that RESTful Rails adds complexity to Rails, which makes it harder to explain how Rails works to newcomers to Rails. I’m very interested to see how this talk is received by the Rails community, so if you weren’t at the talk today, look for it on Confreaks in the next few weeks.

So looking back on RESTful Rails applications that I’ve developed, if they reach even a level of medium complexity, you end up with 50 lines or so in the routes.rb, with at least a handful of nested and custom routes, in addition to all of the resources. Imagine this as the admin interface for a simple blog:

map.namespace(:admin) do |admin| 
  admin.resources :articles, :has_many => :comments, :member => { :publish => :post }
  admin.resources :categories, :has_many => :articles
  admin.resources :comments, :belongs_to => :article
  admin.resources :tags, :has_many => :articles
end

If you are an experienced Rails developer, you can decipher this right away. But is this really easy understand? Or wouldn’t convention over configuration be easier:

map.connect "/admin/:controller/:action/:id"

And what about named routes like new_admin_article_comment_path(@article)? Is that really more clear than admin_path("/comments/new", :article_id => @article)"? Do you really care if the urls are /comments/new?article_id=1 or /articles/1/comments? Maybe I missing something, but I’m starting to like the sound of idea of having almost nothing in my routes and not calling dynamically generated methods everywhere to just to build simple paths. After all, no code is faster than no code.

Posted in Technology | Topics Ruby, Rails, REST | 28 Comments