Summary Do dynamic type systems encourage the proliferation of ad-hoc type checks, and possibly reduce software quality?

Growing Paynes: Twitter's legacy code

Justin Etheredge drew my attention to a recent article in Artima Developer entitled Twitter on Scala, describing some issues that Twitter encountered as it scaled up, and providing justifications for switching the codebase to a different language. While that is interesting in and of itself, what was really thought-provoking is the excerpt from Alex Payne that Justin cites, reproduced in its entirety here:

Alex Payne: I'd definitely want to hammer home what Steve said about typing. As our system has grown, a lot of the logic in our Ruby system sort of replicates a type system, either in our unit tests or as validations on models. I think it may just be a property of large systems in dynamic languages, that eventually you end up rewriting your own type system, and you sort of do it badly. You're checking for null values all over the place. There's lots of calls to Ruby's kind_of? method, which asks, "Is this a kind of User object? Because that's what we're expecting. If we don't get that, this is going to explode." It is a shame to have to write all that when there is a solution that has existed in the world of programming languages for decades now.

– Twitter on Scala. Artima Developer. Published April 3, 2009. Accessed April 7, 2009.

Is Alex unfairly scapegoating Ruby for Twitter's failings, and using this as a reason to move to Scala? Obie Fernandez seems to think so. If the kind_of? checks need to be proliferated because Ruby has framework and language weaknesses, that's one thing. But if it's Twitter legacy code, that's another matter entirely.

In a Twitter exchange between Alex and Obie, Alex admits the problems were with Twitter, not Ruby:

[Alex Payne] » @[Obie Fernandez] Indeed, using kind_of? in that way is "doing it wrong" in Ruby. But as our codebase grew, it became a necessity to combat bugs.

[Obie Fernandez] » @[Alex Payne] Doesn't make sense. Do you mean non-deterministic bugs? An in-depth explanation of the bugs you're talking about would be enlightening

[Alex Payne] » @[Obie Fernandez] Yes, I mean non-deterministic bugs in the giant, legacy, spaghetti parts of our system. Unexpected objects flying around.

It's not very fair to be armchair architects and second-guess Twitter's design decisions with the benefit of hindsight. Nonetheless, I'd be willing to bet that a more thorough analysis of the code base would have revealed the source of the "unexpected objects flying around". That is likely to be the real cause of their issues, not the proliferation of ad-hoc type checks.

But are these perhaps part and parcel of the same underlying problem?

Catching problems

We've stumbled onto an interesting question: Does the use of dynamic type systems defer some of the responsibility that would otherwise be handled by static type checking to the developer in the form of additional required tests? There's no doubt that this is the case. Alex's experience with the Twitter codebase is compelling (albeit anecdotal) evidence for this point.

To see why, consider that the underlying goal of a static type system is to convert as many programming mistakes as possible into type mismatch errors (TME). A TME occurs whenever an expression needs to be of some type T, but its static type is of type U, and there is no automatic way to convert U into T.

Possible TMEs are easy to identify statically. But most static languages will be conservative and reject programs which wouldn't have any issues at runtime. In other words, merely having a candidate for TME can be sufficient to cause a failed compile. For example, the C# compiler will reject a snippet such as the following:

int x = 0;
int y = 1;

if (y == 0) {
  x = "hello!";  // Won't compile, even though it's impossible to reach
}                // this line at runtime and no type error can occur.

In the above example, the attempt to assign "hello!" to x will be a TME: an assignment to x for a static type is required to be of the same type as x or of a type that can be implicitly converted to x. In this case, x is an int, and the expression "hello!" is a string. C# has no implicit conversion from a string to an integer, so failure occurs.

But on closer inspection, we see that this line could never be reached to begin with: the y == 0 condition cannot possibly be true. We're often told to trust the compiler, and let it do its job, because understanding compiler internals and optimizations seems to be trickier than just writing code. But this example seems to go against that; it's pretty clear that this wouldn't really be a problem. Why can't the compiler be smarter than a human in this almost absurdly trivial case?

A tradeoff: flexibility of expression for type safety

The answer is that static type systems try to be as strict as possible to make their analysis easier. In exchange for this strictness, static compilers can make more powerful guarantees about the type safety of their programs at runtime.

Additionally, an assignment like int x = "hello!"; might be considered wrong on its face, since there's a mismatch between what was intended and what was stated. By highlighting such errors at compile time, where they're more easily caught, we save a lot of developer time later down the road. The example we used here is trivial to the point of being a straw man, but in more complex expressions it may not be automatically obvious to a human whether the result is what was intended.

Lost type safety might need to be compensated for elsewhere

Many dynamic type systems give us more powerful constructs to express software than static typing does. It's often quicker to just say what you mean in Python, Groovy, or Ruby and have unit tests sort out the details later.

But when you get around to writing those tests, what do you actually need to test? Certainly, the tests you write for a statically typed language will be similar in some respects to dynamically typed ones. You'll still need to test the logical portions of your software -- for example, whether ComputeSalesTax() computes sales tax correctly, or whether a particular Comparable mixin sorts instances the way you expect. So we wouldn't expect to see much difference between unit tests of static and dynamic languages on this front.

But there's some additional work you need to do with the unit tests of dynamic languages. In general, without checking things yourself, you can't be sure at runtime that a particular object will respond to a particular method signature. More generally, you can't know in advance that it conforms to a particular interface.

That means that, if you really do require the full power of static typing in a dynamic language, you will unquestionably need to do some more legwork in testing to be sure that you didn't miss anything. Each and every time you invoke a method, for example, you will need to scrupulously examine its arguments for compatibility with the method signature. In effect, you will have ultimately implemented static typing for the subset of your software being tested, and in a far more verbose (and likely error-prone) way¹.

This should not be a surprising result: if you want to implement a feature of language L in a different language M using only the provided facilities of M, be prepared for an uphill struggle. This is why, for instance, no one has been successful in getting STM into widespread, mainstream languages: they're not well suited for it.

The real point here is that if you need static typing, you should use a statically typed language. Similarly, if you absolutely need native concurrency primitives, you don't implement them in C#; you use Erlang. Static typing is no different from other language features in this respect.

Best of both worlds: opt-in type systems

As developers, that's frustrating to hear, of course. Why can't we have our cake and eat it too? What if we want the syntactical conveniences of Ruby with the static typing of Java?²

Fortunately, recent additions to mainstream programming languages are beginning to make this sort of thing a reality. For instance, C# 4 will soon have the much-vaunted dynamic pseudokeyword. This essentially functions as a type modifier; expressions of type dynamic have their evaluation deferred to runtime. Effectively, C# now has what I'll call opt-in duck typing.

Even dynamic languages have toyed with the notion of batting for the other team. Guido van Rossum has entertained the idea of static typing in Python at least once before. And ActionScript 3 has optional static typing, an obvious complement to C#'s approach.

You may be able to see how this feature would be valuable in some circumstances -- perhaps you can even identify situations where it'd be useful in your current project. But more importantly, were you to use these opt-in type systems, your unit tests would change accordingly. A dynamic language using opt-in static typing would see some manual type checking disappear for the relevant unit tests, and likewise a static language using opt-in dynamic typing might need a few more checks to make sure messages were going around correctly.

Dynamic unit tests represent your elective use of typing

Dynamic unit tests, then, are like a barometer for measuring how much you care about typing in a particular segment of your software. If you're relatively indifferent to type so long as objects conform to some interface (which is generally the case), then your unit tests will reflect this. This is often the case, which explains why dynamic languages can feel so productive and baggage-free.

But if you'll have difficulty unless the types are exactly right, or if things don't line up just the way you expect, your unit tests will be much richer in assertions and checks. They're essentially functioning as a substitute for as-yet-nonexistent language features like opt-in alternative typing systems.

That, I would surmise, is why Alex and the Twitter crew found themselves a new home with Scala. It wasn't that Ruby was a poor language or that it was ill-suited for development of large-scale software. It's that Scala offered the static typing and rigor that was needed for their particular application. It allowed them to transform those pervasive ad-hoc type checks and rely on the language to provide a more stable type system, one where their assertions could manifest as type declarations instead of ad-hoc typing in unit tests.

At the end of the day, a better product resulted, and that's the goal. Our choice of which language features to use is only tangentially related to the quality of software we produce. But if we can make it easier on everybody with elective language features, and avoid the proliferation of checks that arise when these aren't available, so much the better.

Footnotes

¹ This of course says nothing about the quality of software produced with dynamic or static languages. That has, in my view, less to do with languages and more to do with design decisions and disciplined testing.

² This is really a manifestation of a sort of 80/20 rule: we're getting most of our day-to-day language productivity directly from a relatively small subset of language features. The frustration arises because we want to do that last 20%, but it'd require clumsily accreting stuff from the remaining 80%.

http://www.codethinked.com/post/2009/04/05/Do-We-Create-Type-Systems-In-Dynamic-Languages.aspx