IBM MQ (IBM Message Queue): Overview and Comparison to IronMQ

Wherever two or more people need to wait in line for something, you can guarantee that there will be a queue: supermarkets, bus stops, sporting events, and more.

waiting in line

It turns out, however, that queues are also a useful concept in computer science.

The data structure known as a queue is a collection of objects that are stored in consecutive order. Elements in the queue may be removed from the front of the queue and inserted at the back of the queue (but not vice versa).

Queues are a good choice of data structure when you have items that should be processed one at a time in first-in, first-out (FIFO) order–in particular, the message queue. In this article, we’ll discuss IBM MQ, one of the most popular solutions for implementing message queues, and see how it stacks up against Iron.io’s IronMQ software.

What is a Message Queue?

As the name suggests, a message queue is a queue of messages that are sent between different software applications. Messages consist of any data that an application wants to send, as well as a header at the start of the message that contains information about the data below it.

message in a bottle

Message queues are necessary because different applications consume and produce information at different speeds. For example, one application may sporadically create large volumes of messages at the same time, while another application may slowly process messages, one after another.

The differing speeds at which these two applications operate can pose an issue. Because they are all produced at the same time, all but one of the first application’s messages will be lost by the second application–unless they can be temporarily stored within a message queue.

A message queue is a classic example of asynchronous communication, in which messages and responses do not need to occur at the same time. The messages that are placed on the queue do not require an immediate response, but can be postponed until a later time. Emails and text messages are other examples of asynchronous communication in the real world.

texting

While implementing a basic message queue is a fairly straightforward task, complex IT environments may include communications between separate operating systems and network protocols. In addition, basic message queues may not be resilient when the network goes down, causing important messages to be lost.

For these reasons, many organizations have chosen to use “message-oriented middleware” (MOM): applications that make it easier for different software and hardware components to exchange messages.

There are a variety of MOM products on the market today (like Delayed Job or Sidekiq in the Ruby on Rails world), each one intended for different situations and use cases. In the rest of this article, we’ll compare and contrast two popular options for exchanging data via MOM software: IBM MQ and IronMQ.

What to Consider When Selecting a Message Queue

Message queues are essential to how different applications interact and exchange data within your IT environment. This means, of course, that choosing the right message queue solution is no easy task–picking the wrong one can drastically affect your performance.

Below, we’ll discuss some of the most important factors to consider when choosing a message queue solution.

Features

Depending on the specifics of your IT environment, you may require any number of different features from your message queue. Here’s just a small selection of potential functionality:

  • Pushing and/or pulling: Most message queues include support for both pushing and pulling when retrieving new messages. In the first option, new messages are “pushed” to the receiving application in the form of a direct notification. In the second option, the receiving application chooses to “pull” new messages itself by checking the queue at regular intervals.

pushing on a train

  • Delivery options: You may want to schedule messages at a specific time, or send messages more than once in order to make sure that they are delivered. If so, choose a message queue that includes support for these features.
  • Message priorities: Some messages are more critical or urgent than others. In order to receive the information you need in a timely manner, your message queue may use some way of migrating important messages up the queue (just like letting late passengers cut in front of you at the airport).
  • Persistence: Messages that are persistent are written to disk as soon as they enter the queue, while transient messages are only written to disk when the system is using a large amount of memory. Persistence improves the redundancy of your messages and ensures that they will be processed even in the event of a system crash.

scribe

Scalability and Performance

The more complex your IT environment is, the more difficult it is to scale it all at once. Instead, you can scale each application independently, decoupled from the rest of the environment, and use a message queue to communicate asynchronously.

Certain message queue solutions are better-suited for improving the scalability and performance of your IT environment. Look for options that are capable of handling high message loads at a rapid pace.

Pricing

Different message queue solutions may have different price points and pricing models that lead you to choose one over the other.

“As a service” is currently the predominant pricing model for message queues. This means that customers have a “pay as you go” plan in which they are charged by the hours and the computing power that they use. However, there are also prepaid message queue plans with an “all you can eat” pricing model.

Security

With hacks and data breaches constantly in the news, maintaining the security of your message queue should be a primary concern. Malicious actors may attempt to insert fraudulent messages into the queue and use them to exfiltrate data or gain control over your system.

Message queue solutions that use the Advanced Message Queuing Protocol (AMQP) include support for transport-level security. In addition, if the contents of the message itself may be sensitive, you should look for a solution that encrypts messages while in transit and at rest within the queue.

locked door

What is IBM MQ (IBM Message Queue)?

IBM Message Queue (IBM MQ) is a MOM product from IBM that seeks to help applications communicate and swap data in enterprise IT environments. IBM MQ calls itself a “flexible and reliable hybrid messaging solution across on-premises and clouds.” It includes support for a variety of different APIs, including Message Queue Interface (MQI), Java Message Service (JMS), REST, .NET, IBM MQ Light, and MQTT.

The IBM MQ software has been around in some form since 1993. Thanks to the widespread demand for real-time transactions on the Internet, IBM MQ and other message queue solutions have enjoyed a renewed popularity in recent years.

The benefits of using IBM MQ include:

  • Support for on-premises, cloud, and hybrid environments, as well as more than 80 different platforms.
  • Advanced Message Security (AMS) for encrypting and signing messages between applications.
  • Multiple modes of operation, including point-to-point, publish/subscribe, and file transfer.
  • A variety of tools for managing and monitoring queues, including MQ Explorer, the MQ Console, and MQ Script Commands.

On websites such as TrustRadius and IT Central Station, IBM MQ users mention a few advantages and disadvantages of the software. Some of the recurring themes in these reviews are:

  • IBM MQ is reliable and does its job well, without any lost messages.
  • The software helps to improve data integrity, availability, and security.
  • The user interface may be a little unintuitive and challenging, especially for first-time users.
  • Tools such as MQ Explorer seem to be “aging” and are not as effective as third-party solutions.
  • IBM MQ lacks certain integrations that would be useful in a modern IT enterprise environment.

IBM MQ vs. IronMQ

There’s no doubt that IBM MQ is a robust, mature message queue solution that fits the needs of many organizations. However, it’s far from the only MOM software out there. Offerings such as Iron.io’s IronMQ are highly viable message queue alternatives, and in many cases may be superior to market leaders such as IBM MQ.

What is IronMQ?

IronMQ is a messaging queue solution from Iron.io, a cloud application services provider based in Las Vegas. According to Iron.io, the IronMQ message queue is “the most industrial-strength, cloud-native solution for modern application architecture.”

industrial strength

The software includes support for all major programming languages and is accessible via REST API calls. Iron.io offers a number of different monthly and annual pricing models for IronMQ, ranging from the hobbyist all the way up to the large enterprise.

The benefits of IronMQ include:

  • Support for both push and pull queues, as well as “long polling” (holding a pull request open for a longer period of time).
  • The use of multiple clouds and availability zones, making the service highly scalable and resistant to failure. In the event of an outage, queues are automatically redirected to another zone without any action needed on the part of the user.
  • Backing by a high-throughput key/value data store. Messages are preserved without being lost in transit, and without the need to sacrifice performance.
  • Flexible deployment options. IronMQ can be hosted on Iron.io’s shared infrastructure or on dedicated hardware to improve performance and redundancy. In addition, IronMQ can run on your internal hardware in cases where data must remain on-premises.

IBM MQ vs. IronMQ: The Pros and Cons

Both IBM MQ and IronMQ are cloud-based solutions, which means they enjoy all the traditional benefits of cloud computing: better reliability and scalability, faster speed to market, less complexity, and so on.

Since it was created with the cloud in mind, IronMQ is particularly well-suited for use with cloud deployments. Because IronMQ uses well-known cloud protocols and standards such as HTTP, JSON, and OAuth, cloud developers will find IronMQ exceedingly simple to work with.

software developer

IronMQ users enjoy access to an extensive set of client libraries, each one with easy-to-read documentation. The IronMQ v3 update has also made the software faster than ever for customers who need to maintain high levels of performance.

Customers who already use Iron.io’s IronWorker software for task management and scheduling will find IronMQ to be the natural choice. According to one IronMQ user in the software industry, “I can run my Workers and then have them put the finished product on a message queue – which means my whole ETL process is done without any hassle.”

On the other hand, because it’s part of the IBM enterprise software family, IBM MQ is the right choice for organizations that already use IBM applications. If you already have an application deployed on IBM WebSphere, then it will be easier to simply use it together with IBM MQ.

What’s more, IBM MQ is capable of working well in many different scenarios with different technologies, including mainframe systems. However, some customers report that IBM MQ has a clunky, “legacy” feel to it and is difficult to use in an agile IT environment.

While it’s definitely able to compete with IBM MQ, IronMQ also stacks up favorably against other message queue solutions such as RabbitMQ and Kafka. For example, RabbitMQ’s use of the AMQP protocol means that it is more difficult to use and can only be deployed in limited environments. According to various benchmarks, IronMQ is roughly 10 times as fast as RabbitMQ.

IronMQ Customer Reviews

Of course, reading long lists of software features can only go so far–you need customer feedback in order to make sure that the application really does what it says on the tin.

The good news is that IronMQ has a number of happy customers who are all too eager to share their positive experiences. John Eskilsson, technical architect at the engineering firm Edeva, raves about IronMQ in his testimonial on FeaturedCustomers:

“IronMQ has been very reliable and was easy to implement. We can take down the central server for maintenance and still rely on the data being gathered in IronMQ. When we start up the harvester again, we can consume the queue in parallel using IronWorker and be back to real-time quickly.”

In a review on G2, one user working in marketing and advertising praised IronMQ’s reliability and performance:

“My experience with the message queues was a good one. I had no issues and found the message queues to be very reliable. The website has good monitoring showing exactly what is happening in real time.”

The world’s most popular websites may receive millions of page hits per day, and more during times of peak activity. Businesses such as CNN need a robust, feature-rich, highly available message queue solution in order to get the right information to the right people. CNN is one of many enterprise clients that uses IronMQ as its message queue solution.

IBM MQ vs. IronMQ: Which is Right for You?

At the end of the day, no one can tell you which message queue solution is right for your company’s situation. Both IBM MQ and IronMQ have their advantages and drawbacks, and only one may be compatible with your existing IT infrastructure.

In order to make the final decision, draw up a list of the features and functionality that are most important to you in a message queue. These may include issues such as persistence, fault tolerance, high performance, compatibility with existing software and hardware, and more.

Fortunately, you can also try IronMQ before you buy. Want to find out why so many clients are proud to use IronMQ and other Iron.io products? Request a demo of IronMQ, or sign up today for a free, full-feature 14-day trial of the software.

Securing Serverless

Guy Podjarny published a great blog post discussing the Serverless space from a security perspective. I highly recommend reading it as it touches on some great points, going over both the security benefits and possible risks.

Two points he made definitely stood out to me, and the first was the concept of a greater attack surface. When I explain FaaS (Functions as a Service) to people, many immediately equate a function as being synonymous to a simple API endpoint. To a degree, they are correct. Then, what’s the difference, and why should we look at security in regard to both from different perspectives? I believe the differentiator becomes how the endpoint is exposed, and what its purpose is.

Standard API endpoints will often belong to a broader application or set of microservices that reside behind a shared layer of security. This could be dedicated network hardware, hardened reverse proxies, etc. As a security minded developer, you develop your endpoint and consider the possible client side attack vectors (Guy points to the OWASP Top Ten guide (Open Web Application Security Project) which is a great place to start; Thomas Ptacek also has a great list here); then possibly move on to write another endpoint, which will share these concerns, all the time relying on that first level layer of security.

When you start developing a suite of functions, things can start to get fragmented. Dependencies start to change between functions, software versions might differ, and the ways the functions are triggered may require different configurations on the network/gateway layer.

The second point that stood out was around monitoring: There are countless battle-tested monitoring solutions out there, but the way functions are deployed and used within an underlying architecture might leave them completely out of their scope. Guy makes a great point about how many of these products are agents that rely on long running processes to keep an eye on and collect from. In order to monitor functions, different techniques need to be implemented for short-lived and hot processes.

All of these are great problems to have and point to fast moving innovation in already fast moving industries. You’ll see most vendors and platforms already tackling these issues and building solutions into their products. This space is still young! Here at Iron, we’re committed to helping make IronFunctions become a respected open source solution for delivering FaaS to wherever you want to deploy it.

Four and a Half Years of Go in Production at goto Chicago 2016

GOTO_Chicago2016

Travis Reeder, CTO and co-founder of Iron.io, gave a talk at Goto Chicago 2016 discussing Iron.io’s early migration to Go, why we changed our infrastructure and the benefits it has brought to us.

One of the questions that always comes up after telling people we migrated to Go is:

“Why not Ruby?”

Continue reading “Four and a Half Years of Go in Production at goto Chicago 2016”

Case Study: Astro Digital – DropCam for Space

Astro Digital a case study from space

Thanks to Eutelsat_SA for the base image! CC BY 2.0

Space, the final frontier. If only we could take selfies. Wait, what? We can? This is the service Astro Digital provides. A selfie-stick from space is my crude analogy. Turns out, there are better analogues.

I had the pleasure of catching up with Bronwyn Agrios, head of product at Astro Digital, to learn how it works. In Bronwyn’s (much more refined) words, Astro Digital is kind of like DropCam for space. Just pick a spot to monitor. When new images are snapped, Astro Digital runs some fancy image processing algorithms, and boom! You’re notified of new space-selfies.

Click to enlarge the Astro Digital infographic:

(more content below)

Continue reading “Case Study: Astro Digital – DropCam for Space”

Best Practices and Anti-Patterns for Workers and MQs

Best Practices and anti-patterns for workers and message queues

Thanks to Ruth Hartnup for the base image! CC BY 2.0

If you’ve been programming for a while, it’s probable that someone, somewhere, has recommended the Gang of Four book.

The book dissects Object Oriented programming. It lists numerous ways of royally messing things up, but it’s claim to fame is that it also lists ways to do it right! These well tested paths to success often come with explanations for when to use them and why they’re good at avoiding common pitfalls.

These are design patterns. They’re embedded in the culture of programming, and they’re an amazing way to learn from others’ mistakes. At the outset of any project, a lot of paths are open to you. Design patterns illuminate the dark paths from the healthy, low stress approaches.

Today, we’re releasing our very own set of best practices and anti-patterns in the form of a white paper. It’s a quick read and will save you time as you tinker on your own workers and message queues. So, what do you have to lose?

Continue reading “Best Practices and Anti-Patterns for Workers and MQs”

Docker + Iron.io = Super Easy Batch Processing

lotsocontainers

There is a ton of use cases for batch processing and every business is probably doing it in some way or another. Unfortunately, many of these approaches take much longer than need be. In a world of ever increasing data, the old way can now hinder the flow of business. Some examples of where you’ll see batch processing used are:

  • Image/video processing
  • ETL – Extract Transform Load
  • Genome processing
  • Big data processing
  • Billing (create and send bills to customers)
  • Report generation
  • Notifications – mobile, email, etc.

We’ve all seen something that was created during a batch process (whether we like it or not).

Now, I’m going to show you how to take a process that would typically take a long time to run, and break it down into small pieces so it can be distributed and processed in parallel. Doing so will turn a really long process into a very quick one.

We’ll use Docker to package our worker and Iron.io to do the actual processing. Why Docker? Because, we can package our code and all our dependencies into an image for easy distribution. Why Iron.io?  It’s the easiest way to do batch processing. I don’t need to think about servers or deal with distributing my tasks among a bunch of servers.

Alright, so let’s go through how to do our own batch processing.

Continue reading “Docker + Iron.io = Super Easy Batch Processing”

Shepherding Containers

Shepherding Containers

Thanks to Yahoo for the base image CC BY 2.0

As one of the earliest users of Docker, I’ve had the pleasure of creating and working with multiple different platforms built on containers. Each platform has evolved in step with current ecosystem around it, and I’ve gotten the chance to really put Docker’s “batteries included, but removable” philosophy to the test.

Here at Iron.io, we have launched over 1 billion user containers in production, not to mention the containers we launch to keep our services running. The massive volume of containers we launch is enough to place great demands upon any platform that we use.

In our search for the right direction for the evolution of our platform, we’ve explored as many tools as possible. The release of Docker 1.9, combined with production-ready Docker Swarm and Docker Networking, brings a lot of value to those wanting to roll their own platforms.

Continue reading “Shepherding Containers”

Right-sizing with Docker Stats and cAdvisor

Right-sizing Docker

Thanks to Jared for the base image CC BY 2.0

Containers make life easy. Oh, you don’t have Ruby 2.2 installed? No problem, try this Docker image. Knowing what I tested on my local is exactly what’s running on production gives me warm fuzzies.

Docker gets a lot of love because it simplifies development. That’s not all though. If Docker punished infrastructure, there’d probably be a lot less love going around. Thankfully, Docker does some cool things on the infrastructure side, as well.

The biggest benefit is the “right-sizing” of compute resources. Your program might only need 200 MB of memory. Why dedicate an entire VM + OS to that? Docker insures our compute resources are neatly divided by memory and CPU between instances. Neat! There’s a lot to love about Docker on the infrastructure side, as well.

Continue reading “Right-sizing with Docker Stats and cAdvisor”

Message Queues & Workers: the Heart of Modern Infrastructure

Message Queues + Workers

Thanks to Sonny Abesamis for the base image! CC BY 2.0

Increasingly, message queues and workers are intertwined with the language of modern infrastructure. You might rely on explicit solutions like IronMQ or IronWorker. You might not. Whether you do or don’t is irrelevant: MQs and workers are in everything these days.

MQs and workers are hidden heroes, quietly powering a lot of the technology that many of us rely on. They’re core components in programming languages, MVC frameworks, and even web servers.

As a result, when making infrastructure decisions a good understanding of both MQs and workers is essential. The white paper below will take you from a fuzzy understanding to a well-versed conceptualization for both MQs and workers.

Download the latest white paper from Iron.io: a Refresher on Message Queues & Workers.
Give it a read and let us know what you think!

Helpful hints: Pre-commit

Get going with pre-commit (video)

I had a lot of fun sharing pre-commit with everyone earlier this week. In case you missed it, here’s a link. As a follow up, I whipped together a quick How-To video.

In it, you’ll follow along as I add a pre-commit hook to a brand new Node.js project. Init a git repo locally, and you can the same as you watch. I’ll take you through adding the initial config, picking your hook, and testing to make sure pre-commit works. Continue reading “Helpful hints: Pre-commit”