Getting to “go”

Despite the title, this isn’t another sales excellence screed, travel policy dictat, or a washroom deconfliction methodology. Rather, this is about our journey to select a new programming language and platform for our performance-sensitive services.

Starting Point

We began with a set of web services successfully built on the Ruby/Rails ecosystem. Given an extremely short timeframe to deliver version 1 and that the majority of the early product requirements were focused on the user interfaces, Ruby made a lot of sense.

As the scaling requirements of some of our services grew beyond the initial prototype, we found (to little surprise) that Ruby MRI just couldn’t keep up. We did some investigation and benchmarking with JRuby, hoping that it would offer a clean transition to a more performant services platform. Not so much… so we found ourselves at a crossroads.

Picking A New Path

Choosing a new programming language often bears more resemblance to a religious debate than to a purely logical process. Each of us has our blacklist (“Java is too verbose”, “Python’s dynamic typing makes you lazy”, “Scala is too hard to learn”), our hot list (“Ooh, let’s try Racket”, “Haskell looks like fun”, “Why not Erlang?”), and our unique perspective (embedded, web apps, enterprisy, whatever). Despite all of this, we intentionally took a more balanced and reasoned approach.

Given our small team size and aggressive schedule, we simply couldn’t afford to evaluate every possible platform. Instead, we immediately narrowed the field down to a reasonable set of candidates by ignoring some less likely scenarios:

• Clojure/Haskell/Erlang – too esoteric, not enough industry traction, too slow for certain workloads
• C/C++/Objective-C – not enough abstraction, very complex syntax
• Python – same architectural deficiencies as Ruby

We knew this effort would require extensive research, so we wanted to focus our energy by first defining a detailed evaluation criteria. Obviously, we demanded that our new platform provide an order of magnitude increase in performance (at least!) along with very strong support for concurrency and parallelism. In addition, we also wanted to understand:

• ramp-up – How quickly can people learn the platform? What kind of safeguards does the platform have to inhibit new people from implementing bad code?
• open source and community involvement – What does the open source community look like? How mature are the “gem” equivalents? Do these open source components use best practices around testing, like rubygems?
• TDD/BDD – What tools are in place to support TDD/BDD? Is there something like RSPEC for the platform? Do strong mocking tools exist?
• build time and effort – What effort is involved in the build? Does the build time slow down as the source code becomes larger and more complex?
• commercial adoption – What are other folks in our industry using?
• tooling – What tools exist for developing (IDEs), testing, documenting, building, profiling, deploying, and monitoring? What about static analysis?
• scalability – How easily can the platform scale horizontally and vertically?
• recruiting – How likely will we be able to attract and excite new developers?

We avoided the approach of writing small toy programs in each language to attempt to extrapolate conclusions. We felt that these quick “Hello World” spikes didn’t provide enough exposure to grok the nuances and pain points and we specifically avoided deriving conclusions from them.

After defining our evaluation criteria we were off to the races evaluating the platforms! Those that remained on our short list were Java, Scala, Go, and D.

Java was the most well understood platform since several clypd senior developers had deep experience with that platform. Java received strong marks for ramp-up, community, testing, tooling, and commercial adoption. Unfortunately, Java’s deficiencies quickly doomed its selection as the same senior developers didn’t even advocate for it. Their concerns included code verbosity, long build times, and a lack of a consistent strategy for concurrency.

None of the developers had any previous exposure to D besides the occasional thread from Andrei @ Facebook. Although D appears to address many of the shortcomings in Java and C++, ultimately we were challenged finding enough public information – anecdotes, critiques, and analysis – to give D anything but failing marks for community and adoption.

The final two remaining platforms, Scala and Go, both scored high in many areas. Both have

• strong concurrency models
• active communities and existing commercial installations
• proven scalability
• positive trending to help with recruiting

Scala’s functional model appealed in principle, but the language’s flexibility/complexity and associated ramp-up required by the team was a significant concern. Further, Scala’s lack of stability, ridiculously long build times (too long to support TDD), and clunky build tools are in stark contrast to Go’s simplicity, 1.x compatibility guarantee, and blazingly fast build times. Lastly, we’d heard some mixed feedback from folks in our network who were already using Scala. Some loved it, some didn’t. As a team, we met one final time to review our findings and recommend a path forward. After a contentious but educational process, we could finally say that we knew which way to Go. [groan]

Stay tuned for more updates as we learn about Go, its ecosystem and the Gopher community.

Brian Fallik is Principal Developer and Joel Melby is the CTO at clypd.