GoLangSF April 2016: Gohan, Essentier and rqlite

Thank you to New Relic for hosting the April 2016 Golang SF meetup, sponsored by Iron.io!

Gohan with Nachi Ueno

Gohan provides a REST-based API server, database backend, CLI, and WebUI generated from a JSON schema. Gohan can interpret definitions and act as a REST API server. As opposed to Open API, which defines the API, Gohan defines the controller and data model as well.

Gohan provides for inheritance, schemas, policies and extensions. You can use a YAML format. Permission are defined by property using a CRUD model. You can also specify the relationship in the property;

Extensions

Gohan supports custom logic using Go, JavaScript, or Gohan DSL. Gohanscript is Ansible-inspired, and easy to read and write for non-programmers. Unfortunately, it’s a bit slow right now. The JavaScsript extensions are Otto-based and are a bit slow. Go extensions are also possible and quite fast, but they must be compiled.

Policies & Integrations

Gohan supports RBAC-based policy enforcement for your API. Additionally, Gohan can integrate with 3rd-party system using Sync (etcd) and OpenStack Keystone.

Nachi Ueno (@nati) is a Distinguished Engineer at Juniper Networks and is leading key OpenStack/cloud deployments and OpenContrail development efforts. Nachi has been an active contributor to the OpenStack community since 2010.

Essentier with Char Wu

Unit tests are good for your project; they are repeatable, self-checking and dependable. Unit tests are also fine-grained, which can be good or bad. You need something else when the scope of your testing exceeds the function or the strut. You can use mocks and stubs to obfuscate the dependencies of your code, but you will lose fidelity. If you make a mistake in your mocks, or even if your mock is perfect, it will be a leaky abstraction. This can cause errors higher up on your testing stack.

Integration testing to the rescue

This is why we have integration tests. If unit testing could do the job of integration tests, we would not need integration testing. It solves the granularity issue of unit tests by allowing us to test at a larger scope. However, these tests tend to be very slow, and tend to be fairly coarse-grained, testing the entire application end-to-end. We want high-fidelity tests of any granularity.

Source services are those you write yourself, and you own the code. These services need to be built by the dockerfile used by Nomock. Built services are already built and packaged into docker images, so you just need to specify the docker image in the configuration file. Finally, you have deployed services, where you already have the service up and running, but you need to direct your testing framework to the running service. The library Spickspan will allow you to do platform-agnostic service discovery with pluggable providers.

Essentier

Essentier Nomock is a testing framework where integration tests look just like usual unit tests except that there is no need to mock out any dependency. When the tests run, source code will be Git pushed to the cloud and all the components under test will be built, packaged and deployed in parallel for maximum speed. The end result is unit-like, high fidelity testing at any granularity. What’s even better is that after the integration tests pass, the project along with its dependencies can be deployed directly to a cloud such as the Google Container Engine without any code changes.

Chaur Wu is founder of Essentier. He is a professional software developer for more than 15 years and has been programming in Go for 2 years. In addition he designed a puzzle game based on lambda calculus. He is author of the book “Pro DLR in .NET 4” and co-author of the books “Professional Design Patterns in VB.NET” and “Professional UML with Visual Studio .NET”.

Talk 3: rqlite with Philip O’Toole

rqlite is a distributed relational database, which uses SQLite as its storage engine. rqlite gives you the functionality of a robust, fault-tolerant, replicated relational database, but with very easy installation, deployment, and operation. rqlite is written in Go and uses Raft to achieve consensus across all the instances of the SQLite databases. As you change the SQLite files on one machine, the files on other machines change as well. rqlite ensures that every change made to the database is made to a quorum of SQLite files, or none at all.

Why SQLite?

We had a need for a lightweight database, replicated among many machines. I decided to see if I could do this using Go. SQLite is a rock-solid production database. With replication, you get reliability. Thanks to Go, you get a binary that is very easy to deploy.

A quick side-note: Raft

Raft is a distributed consensus protocol that allows us to build clusters of servers such that a quorum of servers, each server has the same state.

rqlite nodes

Each rqlite node is a stack that works its way up from sqlite, through Raft and HTTP, to the clients.

rqlite clusters are most practical when 3, 5 or 7 nodes in size. Even numbers don’t get you anything, and larger than 9 is unwieldy. And by the same token, rqlite is not suitable for replicating massive clusters.

Philip O’Toole is Director of Data Platform Engineering at Percolate, a marketing SaaS. Prior to joining Percolate he was Director of Engineering, and part of the database development team, at InfluxData. You can find him online at https://www.philipotoole.com.