Testing in a statically-typed language

December 19, 2008

One of the many "fringe" languages that has been on my radar for some time now is Haskell. Haskell has what looks to be a pretty good book available online and a local book club forming, which presents a great opportunity to study it. What is interesting to me about Haskell is that it is a statically-typed language, like Java, not a dynamically-typed language like Lisp, Ruby, Python, JavaScript, etc. I have really become fond of dynamic typing while programming in Ruby, but I always like studying languages that challenge your fundamental beliefs about programming.

Incidentally, this is one of the reasons I have been really interested in Clojure. It challenges the validity of object-oriented project in a very serious way. In Clojure, you don't define new classes, instead your program is simply comprised of functions that takes input, possibly call other functions, and return values. Studying functional languages helps you to see the benefits and weaknesses of object-oriented programming. Read more about this idea of studying programming in Glenn Vanderburg's article about Koans.

So anyway, back to Haskell, and the paradigm here that is worth studying is static typing versus dynamic typing. I must admit, after moving from Java (a statically-typed language) to Ruby (a dynamically-typed language), I'm currently a big fan of dynamically-typed languages. So I'm trying to keep an open mind in studying this, but this passage from Chapter 2 of Real World Haskell I have to object to:

Programs written in dynamically typed languages require large suites of tests to give some assurance that simple type errors cannot occur. Test suites cannot offer complete coverage: some common tasks, such as refactoring a program to make it more modular, can introduce new type errors that a test suite may not expose.

In Haskell, the compiler proves the absence of type errors for us: a Haskell program that compiles will not suffer from type errors when it runs. Refactoring is usually a matter of moving code around, then recompiling and tidying up a few times until the compiler gives us the "all clear".

One thing I learned when moving from Java to Ruby is that developers often rely on the compiler as a false sense of security. The notion that you can refactor a bunch of code, get it to the point where it compiles and then feel that your code works is dangerous. You code can still contain all sorts of runtime and logic errors, therefore if you want to have any degree of certainty that your code is free of bugs, you need to have some sort of test suite, regardless of if you are using a dynamic and statically typed language.

I've also heard the argument made that statically-typed languages allow you to write less tests, but I have also not found this to be the case. Most syntax and type errors will be found with the same tests you use to test the logic of the program. In other words, you don't have to write one suite of tests to check for syntax errors, one for type checking and another for testing the logic of your program.

So the moral of the story is that regardless of whether you are writing your code in a statically or dynamically typed language, you still have to TATFT.

Posted in Technology | Tags DynamicTyping, Haskell, StaticTyping, Ruby, Java, Testing, Clojure

Comments

1.

What static language people don't get about dynamic languages is that in dynamic languages you test what you *need* and ignore all the corner cases that a static language would catch but which don't matter because you're not going to hit them anyway. Also, that unit tests go deeper, because they test typing and semantics at the same time.

What dynamic people don't get about static languages is that in a *good* static language the standard testing guideline "test your code, don't test the framework" can be extended so that much more of your code falls under the category of "the framework" and hence doesn't need to be tested.

# Posted By Avdi on Saturday, December 20 2008 at 12:08 AM

2.

"I've also heard the argument made that statically-typed languages allow you to write less tests, but I have also not found this to be the case."

I'm very interested in this topic. Do you have some numbers for your claim?

There are some numbers around from David Pollak concerning Ruby and Scala:

"We discovered that anything shy of 95% code coverage with Rails means that type-os turn into runtime failures."

Peace
Stephan

Programming is hard - http://www.codemonkeyism.com

# Posted By Stephan Schmidz on Saturday, December 20 2008 at 4:18 AM

3.

"I've also heard the argument made that statically-typed languages allow you
to write less tests, but I have also not found this to be the case."

Is this based on your previous work with Java or do you have some experience
with expressive type systems like Haskell's, OCaml's or SML's? You cannot
transpose conclusions drawn from the use of Java to these languages, as its
type system is very limited.

You also say that "most syntax and type errors will be found with the same
tests you use to test the logic of the program", which again seems to stem
from the very limited capabilities of Java's type system, as it conveys the
idea that type errors are necessarily trivial, to the point that they can
almost be likened to "syntax errors".

A good type system allows you to encode useful system-wide invariants which
will be enforced systematically by the compiler and are sometimes hard to
test, e.g., "the user is not authenticated and thus has read-only access
throughout the subsystems" or "the generated markup is guaranteed to be valid"
(if not, the code will not type/compile) vs. "it happens to be valid in this
or that case" (unit testing).

# Posted By 88.6.2.29 on Saturday, December 20 2008 at 5:40 AM

4.

I've eventually come to realise that when Haskell advocates use the phrase "type", they don't mean the "Is it a float? Is it an int?" stuff that you're used to, they mean any property of a program that can be verified by the compiler. Intermediate-to-advanced Haskell usage appears to be all about identifying such properties and encoding them into the type system. An easy example would be string escaping in code to generate HTML: you have a type EscapedString and a function escape :: String -> EscapedString, and declare all your HTML-outputting functions to only accept EscapedStrings. Thus, failure to escape a string becomes a "type error". In more complex cases, I have no idea how they find these invariants, and still less of an idea how they ensure they've found enough: I'm currently trying to pin dons down on the subject in this reddit thread.

But anyway, Haskell has not one but two test frameworks, HUnit and QuickCheck, so at least some Haskellers see the need for testing :-). HUnit is what you're used to, but QuickCheck's a bit more interesting.

# Posted By pozorvlak on Saturday, December 20 2008 at 6:07 AM

5.

The point of that excerpt from the book is not that all testing goes away. Sure, refactoring may introduce tons of logic bugs, but in a statically typed language, most, if not all, type bugs will not exist at runtime. And refactoring many times includes changing the type of an interface parameter. I think dynamically typed lanugages have there place, I'm a big fan of clojure. I just don't see how large projects with lots of collaborators, could be practical without static typing. I think smaller projects with small teams are where dynamic languages make sense.

# Posted By Mark P on Saturday, December 20 2008 at 10:37 AM

6.

Hey Guys,

Thanks for the feedback. I'm definitely interested in learning more about what the Haskell type system has to offer, so I'll try to avoid saying anything disparaging about static typing until I've have a better understanding of how it works in Haskell. I'm definitely willing to admit the type system in Haskell is probably more powerful that the type system in Java, which I believe is the only other statically typed language I've had any significant experience with.

# Posted By Paul Barry on Saturday, December 20 2008 at 1:58 PM

Comments Disabled