What’s with the Docker community’s love affair with Alpine Linux? Tiny containers means more compute resources left over for actual… computing! Alpine Linux is particularly tiny. It says so, right on the tin: “Alpine Linux is a security-oriented, lightweight Linux distribution based.”
Do you like saving money? I like saving money. Better resource utilization means happy bank accounts.
Let’s take a closer look at Alpine Linux on Docker. Heck, while we’re at it let’s build our own image.
Exosphere champions best of breed cloud applications. In their own words, “We’ve set out on a mission, a quest if you will, to gather together the best small to medium applications in each class, and try to bolt them together in such a way that combined they form a powerful, user-friendly, complete core small business package.”
For the gnarly job of data synchronization exosphere found few options. Solutions for piping core application data certainly exist, but most vendors lock you down like lawn furniture. Exosphere found IronWorker appealing, since it saved them the hassle of building their own out of the box solution.
Exosphere is built on a Node.js + Windows development stack. Today, they’ve agreed to let us share their recent post on getting IronWorker + Windows + Node.js humming in unison.
If you’re curious what twists and tweaks are required to get IronWorker going on Windows, read on!
There is a ton of use cases for batch processing and every business is probably doing it in some way or another. Unfortunately, many of these approaches take much longer than need be. In a world of ever increasing data, the old way can now hinder the flow of business. Some examples of where you’ll see batch processing used are:
ETL – Extract Transform Load
Big data processing
Billing (create and send bills to customers)
Notifications – mobile, email, etc.
We’ve all seen something that was created during a batch process (whether we like it or not).
Now, I’m going to show you how to take a process that would typically take a long time to run, and break it down into small pieces so it can be distributed and processed in parallel. Doing so will turn a really long process into a very quick one.
We’ll use Docker to package our worker and Iron.ioto do the actual processing. Why Docker? Because, we can package our code and all our dependencies into an image for easy distribution. Why Iron.io? It’s the easiest way to do batch processing. I don’t need to think about servers or deal with distributing my tasks among a bunch of servers.
Alright, so let’s go through how to do our own batch processing.
A few months ago Adrian Cockroft gave a keynote at devopsdays Amsterdam. In it, he recommended the use of Gauntlt. It’s a robust ruggedization framework for APIs. In plain English, Guantlt is an easy way to test for common security issues.
Have you started using Gauntlt yet? Me neither. Cockroft is sad. Let’s change that.
It turns out, Gauntlt is just the type of tool that IronWorker is great for. It’s modular, and easy to parallelize with the worker pattern. This post is a guide for getting your own Gauntlt up and running.
I’ve often coded in PHP. Recently, I realized that when I code in Go, it’s easier to spot my PHPisms. After weeks of cleaning them up, it only takes a day to fry my brain with them again. I’m sure I’m not the only developer doing this, so I started collecting tips and tricks to help with the process. I’m sure other ElePHPants have their own tips and tricks, but here are a few of mine.
You don’t always need an additional memory allocation.
PHP’s API is formatted into functions that return values. Want to parse JSON? Just read a string into a variable and json_decode it.
I know – there are exceptions like fputcsv and fgetcsv. Still, you can anticipate that you pass a variable as input and get some kind of output. Variables, as opposed to resources, must be noted here.
Conversely, Go allows for variables as much as streams. You’re able to json.Marshal or json.(*Encoder).Encode an io.Reader.
In Go, you have to determine what to do and the ramifications of your actions. Will it be better to process loading/unloading from memory (json.Marshal) or just iteration through the io.Reader (json.Encoder)?
Containers make life easy. Oh, you don’t have Ruby 2.2 installed? No problem, try this Docker image. Knowing what I tested on my local is exactly what’s running on production gives me warm fuzzies.
Docker gets a lot of love because it simplifies development. That’s not all though. If Docker punished infrastructure, there’d probably be a lot less love going around. Thankfully, Docker does some cool things on the infrastructure side, as well.
The biggest benefit is the “right-sizing” of compute resources. Your program might only need 200 MB of memory. Why dedicate an entire VM + OS to that? Docker insures our compute resources are neatly divided by memory and CPU between instances. Neat! There’s a lot to love about Docker on the infrastructure side, as well.
Microservices are great! Or, at least that’s what the internet keeps telling me. There are a lot of upsides, but there are more than a few challenges too.
For example, microservices are a polyglot’s dream. Have a Rails app and a use case where Ruby seems a bit too slow? No problem, microservices and Go are a good starting point.
As you might imagine, it’s a nightmare keeping track of style guides and best practices on a per service basis. If only there were a tool to help manage this mess. Oh, hey there pre-commit!
Let’s set the stage first… imagine we operate a windmill farm and want to continually track optimal weather conditions to maximize energy output. What basic steps need to be taken?
Sensors capture surrounding weather conditions
Captured data is delivered to a backend service
The service calculates the expected power generation
Calculated data is delivered to an analytics system
Data is presented in a variety of charts and maps
This process flow sounds similar to the common Extract, Transform, Load (ETL) pattern, however the distinction to make here is that data is pushed from the source to the backend service instead of pulled. This means we need to update our pipeline to be more reactive.
Howdy! I’m Paul, and I’m one of the newer faces here at iron.io. My plan is to blog, champion local meetups, and empower developers. That last one is a whiz-bang business term for “help folks learn how to do dead-simple job processing.”
Today, I’ll blog about an old love of mine, Common Table Expressions (CTEs). It’s a tool anyone who’s even sneezed near a SQL statement ought to know about. Just a select few SQL sorcerers seem to be aware of their existence. Let’s change that! Continue reading “Legible + efficient SQL with CTEs”