How To Build Your Own Docker Images

Build Your Own Docker Images

Thanks to Ugur Ceylan for the base image! CC BY 2.0

What’s with the Docker community’s love affair with Alpine Linux? Tiny containers means more compute resources left over for actual… computing! Alpine Linux is particularly tiny. It says so, right on the tin: “Alpine Linux is a security-oriented, lightweight Linux distribution based.”

Do you like saving money? I like saving money. Better resource utilization means happy bank accounts.

Let’s take a closer look at Alpine Linux on Docker. Heck, while we’re at it let’s build our own image.

Continue reading “How To Build Your Own Docker Images”

Running IronWorker on Docker + Node.js + Windows

docker-nodejs-windows

Exosphere champions best of breed cloud applications. In their own words, “We’ve set out on a mission, a quest if you will, to gather together the best small to medium applications in each class, and try to bolt them together in such a way that combined they form a powerful, user-friendly, complete core small business package.”

For the gnarly job of data synchronization exosphere found few options. Solutions for piping core application data certainly exist, but most vendors lock you down like lawn furniture.  Exosphere found IronWorker appealing, since it saved them the hassle of building their own out of the box solution.

Exosphere is built on a Node.js + Windows development stack. Today, they’ve agreed to let us share their recent post on getting IronWorker + Windows + Node.js humming in unison.

If you’re curious what twists and tweaks are required to get IronWorker going on Windows, read on!

Continue reading “Running IronWorker on Docker + Node.js + Windows”

Docker + Iron.io = Super Easy Batch Processing

lotsocontainers

There is a ton of use cases for batch processing and every business is probably doing it in some way or another. Unfortunately, many of these approaches take much longer than need be. In a world of ever increasing data, the old way can now hinder the flow of business. Some examples of where you’ll see batch processing used are:

  • Image/video processing
  • ETL – Extract Transform Load
  • Genome processing
  • Big data processing
  • Billing (create and send bills to customers)
  • Report generation
  • Notifications – mobile, email, etc.

We’ve all seen something that was created during a batch process (whether we like it or not).

Now, I’m going to show you how to take a process that would typically take a long time to run, and break it down into small pieces so it can be distributed and processed in parallel. Doing so will turn a really long process into a very quick one.

We’ll use Docker to package our worker and Iron.io to do the actual processing. Why Docker? Because, we can package our code and all our dependencies into an image for easy distribution. Why Iron.io?  It’s the easiest way to do batch processing. I don’t need to think about servers or deal with distributing my tasks among a bunch of servers.

Alright, so let’s go through how to do our own batch processing.

Continue reading “Docker + Iron.io = Super Easy Batch Processing”

Adrian Cockroft Wants You to Run Gauntlt

gauntlt ironworker simple security

Thanks to David Goehring for the base image CC BY 2.0

Secure APIs with a robust ruggedization framework

A few months ago Adrian Cockroft gave a keynote at devopsdays Amsterdam. In it, he recommended the use of Gauntlt. It’s a robust ruggedization framework for APIs. In plain English, Guantlt is an easy way to test for common security issues.

Have you started using Gauntlt yet? Me neither. Cockroft is sad. Let’s change that.

It turns out, Gauntlt is just the type of tool that IronWorker is great for. It’s modular, and easy to parallelize with the worker pattern. This post is a guide for getting your own Gauntlt up and running.

Continue reading “Adrian Cockroft Wants You to Run Gauntlt”

Best practices for ElePHPants coding in Go

Coding in PHP to GO

Thanks to Guido da Rozza for the base image CC BY 2.0

I’ve often coded in PHP. Recently, I realized that when I code in Go, it’s easier to spot my PHPisms. After weeks of cleaning them up, it only takes a day to fry my brain with them again. I’m sure I’m not the only developer doing this, so I started collecting tips and tricks to help with the process. I’m sure other ElePHPants have their own tips and tricks, but here are a few of mine.

You don’t always need an additional memory allocation.

PHP’s API is formatted into functions that return values. Want to parse JSON? Just read a string into a variable and json_decode it.

I know – there are exceptions like fputcsv and fgetcsv. Still, you can anticipate that you pass a variable as input and get some kind of output. Variables, as opposed to resources, must be noted here.

Conversely, Go allows for variables as much as streams. You’re able to json.Marshal or json.(*Encoder).Encode an io.Reader.

In Go, you have to determine what to do and the ramifications of your actions. Will it be better to process loading/unloading from memory (json.Marshal) or just iteration through the io.Reader (json.Encoder)?

Continue reading “Best practices for ElePHPants coding in Go”

Right-sizing with Docker Stats and cAdvisor

Right-sizing Docker

Thanks to Jared for the base image CC BY 2.0

Containers make life easy. Oh, you don’t have Ruby 2.2 installed? No problem, try this Docker image. Knowing what I tested on my local is exactly what’s running on production gives me warm fuzzies.

Docker gets a lot of love because it simplifies development. That’s not all though. If Docker punished infrastructure, there’d probably be a lot less love going around. Thankfully, Docker does some cool things on the infrastructure side, as well.

The biggest benefit is the “right-sizing” of compute resources. Your program might only need 200 MB of memory. Why dedicate an entire VM + OS to that? Docker insures our compute resources are neatly divided by memory and CPU between instances. Neat! There’s a lot to love about Docker on the infrastructure side, as well.

Continue reading “Right-sizing with Docker Stats and cAdvisor”

Polyglot microservices need pre-commit + git

pre-commit plus git equals good for microservices
Thanks to Jeremy Keith for the base image! CC BY 2.0

 

Microservices are great! Or, at least that’s what the internet keeps telling me. There are a lot of upsides, but there are more than a few challenges too.

For example, microservices are a polyglot’s dream. Have a Rails app and a use case where Ruby seems a bit too slow? No problem, microservices and Go are a good starting point.

Sounds great! However, coding in multiple languages brings some hurt, too. A new language means a new style guide. Work at a shop with both a front end Javascript team and a backend Node.js group? Teams working in different domains are likely to enforce their own domain specific best practices.

As you might imagine, it’s a nightmare keeping track of style guides and best practices on a per service basis. If only there were a tool to help manage this mess. Oh, hey there pre-commit!

Continue reading “Polyglot microservices need pre-commit + git”

E is for Event: A Fresh Take on ETL

ETL

As a follow up to my previous post, The Workloads of the Internet of Things, I wanted to walk through a real world example that fully captures the principles of event-driven computing put forth.

Let’s set the stage first… imagine we operate a windmill farm and want to continually track optimal weather conditions to maximize energy output. What basic steps need to be taken?

  1. Sensors capture surrounding weather conditions
  2. Captured data is delivered to a backend service
  3. The service calculates the expected power generation
  4. Calculated data is delivered to an analytics system
  5. Data is presented in a variety of charts and maps

This process flow sounds similar to the common Extract, Transform, Load (ETL) pattern, however the distinction to make here is that data is pushed from the source to the backend service instead of pulled. This means we need to update our pipeline to be more reactive.

Continue reading “E is for Event: A Fresh Take on ETL”

Legible + efficient SQL with CTEs

Thanks to Jonny Hughes for the image! Thanks to Jonny Hughes for the base image!

Howdy! I’m Paul, and I’m one of the newer faces here at iron.io. My plan is to blog, champion local meetups, and empower developers. That last one is a whiz-bang business term for “help folks learn how to do dead-simple job processing.”

Today, I’ll blog about an old love of mine, Common Table Expressions (CTEs). It’s a tool anyone who’s even sneezed near a SQL statement ought to know about. Just a select few SQL sorcerers seem to be aware of their existence. Let’s change that! Continue reading “Legible + efficient SQL with CTEs”

An Easier Way to Create Tiny Golang Docker Images

Docker_Golang_IronAtlassian just posted a blog post about creating Static Go binaries with Docker on OSX which shows how to make a tiny little Docker image (~6MB total) with your Go program inside. When I read it, I thought “this is cool… but way too complicated”. I’ve been using Docker a lot lately and knew there had to be an easier way. Here is that way. Continue reading “An Easier Way to Create Tiny Golang Docker Images”