Go After 2 Years in Production

By Iron | August 27, 2013 |

After running Go for two years in production at Iron.io, I wanted to share our experience/feelings about it. We were one of the first companies to use Go (golang) in production and we didn't know what to expect in the long run, but so far, so great.

I talked a little about about this in a previous post about switching to Go from Ruby, but this will go into specific things that we love about the language, the things we learned along the way. In no specific order, here they are:

Performance
Memory
Concurrency
Reliability
Deployment
Talent

Table of Contents

Performance

When we were first deciding on what language we were going to use, we did some research and created some mock tests for our use case, a message queue. I wrote a partial beanstalkd clone in Go that spoke the beanstalkd protocol so I could use existing clients to test it. Go performed very well, almost the same as the official C version (and it was surprisingly easy to write).

You can find benchmarks comparing Go to a bunch of other languages at the Computer Language Benchmarks Game site. The graph below compares it to Java (which would probably be the language we'd be using if Go didn't exist):

That benchmark shows that Go is a bit faster in a few cases, but slower in the rest. Still, not too bad for a language that is only a few years old. I have no doubt it will catch up given some time. You can also see that the memory footprint is substantially better though.

Just for fun, check out the differences between Go and Ruby below, it’s insanely better in both performance and memory usage.

Two years in, Go has never been our bottleneck, it has always been the database.

Memory

Go doesn't have a virtual machine or interpreter to load up, so it starts fast and small. IronMQ starts running in ~6444 KB of resident memory including loading configs, making connections, etc. After it’s been running for a while, it increases due to caching and what not. Right now, our production servers are running at ~400 MB (which I realize is somewhat irrelevant as this will depend on your application).

Two years in, we've never had a memory leak or problems related to memory.

Concurrency

Concurrency is a huge part of Go with some high level constructs that make it a breeze to use. I used Java for many, many years and got very comfortable with the java.util.concurrency package, which is a pretty nice set of concurrency tools, but it's just not the same as Go in terms of simplicity or the underlying implementations. Go has goroutines for concurrent operations and channels for communicating between them. Goroutines are particularly interesting:

"Goroutines are part of making concurrency easy to use. The idea, which has been around for a while, is to multiplex independently executing functions—coroutines—onto a set of threads. When a coroutine blocks, such as by calling a blocking system call, the run-time automatically moves other coroutines on the same operating system thread to a different, runnable thread so they won't be blocked. The programmer sees none of this, which is the point. The result, which we call goroutines, can be very cheap: unless they spend a lot of time in long-running system calls, they cost little more than the memory for the stack, which is just a few kilobytes. To make the stacks small, Go's run-time uses segmented stacks. A newly minted goroutine is given a few kilobytes, which is almost always enough. When it isn't, the run-time allocates (and frees) extension segments automatically. The overhead averages about three cheap instructions per function call. It is practical to create hundreds of thousands of goroutines in the same address space. If goroutines were just threads, system resources would run out at a much smaller number." Source

One thing we had to do here was limit concurrency to make sure we didn't overload our database or other services during spikes. We did this with a simple semaphore using Go channels.

Reliability

Reliability in a language is kind hard to quantify, but we've found our Go applications to be very robust. I don't think we've had a failure/crash that wasn't related to some external problem (read: database or some bad library). In general though, there are very high quality open source libraries out there for Go. We've found no memory leaks and no major core library bugs.

I even find that our code is higher quality just due to the fact that it's written in Go. I'm not entirely sure why this is, but I get a warm and fuzzy feeling about our stuff written in Go. Perhaps it's the very strict compiler that even forces us to remove imports and variables that aren't in use. Perhaps it's the small amount of code you have to write to get a lot done. Maybe I'll figure this out and write more about it some day.

Deployment

Go compiles into a single, static binary file so deployment is simply putting the file on a server and starting it up. No dependencies required. No runtime required (you don't need to install Go on the server). And it's small; the IronMQ binary is ~6MB.
Rolling Back

If something goes wrong after deploying and you need to roll back, you can just stop the bad process then start the previous binary. You don't need to worry about a dependency being upgraded since the entire program is compiled into a single binary.

Talent

We took a big risk choosing Go when there wasn't a lot of people that knew the language, let alone people who had heard of it. We were the first company to post a Go job on the golang nuts mailing list and we were taken aback by the quality of people that applied. We received applications from developers at some of the top tech companies with tons of experience, some with PhD's, working on some hardcore projects. Most weren't programming full-time in Go, but had worked with it enough to be proficient and could transfer their experience and knowledge over. I'm not sure it even mattered what we were trying to build, they just wanted to work with Go.

Our first Go hire was one of the core Golang developers, +Evan Shaw, who has been with us ever since.

Conclusion

I can confidently say that after two years working with Go, we made the right choice. If we had started Iron.io today, it would have been a no brainer to choose it. A lot of other companies are using it now too including Heroku and Google and the people I talk to about it all have similar opinions. +Rob Pike, one of the creators of Go said:

“We realized that the kind of software we build at Google is not always served well by the languages we had available,” Pike said in 2011. “Robert Griesemer, Ken Thompson, and myself decided to make a language that would be very good for writing the kinds of programs we write at Google.”

+Derek Collison, the founder of Apcera, said recently in a Wired article:

“The management layers and infrastructure layers of the newer technologies that provide this cloud delivery model?” he tells Wired. “Within two years, a majority will be written in Go.”

Is Go the next gen language we've been waiting for. It's a bit too early to say, but it's certainly off to a good start.

-------

If you're in the bay area and interested in learning about Go, come join us at the next GoSF meetup.

Feel free to contact/follow me on g+: +Travis Reeder .

UPDATE: Lots of discussion at Hacker News and Reddit.

Posted in Serverless News and tagged Go

About Iron

View all posts by Iron →