Blog

Search This Blog

Loading...

Thursday, February 28, 2013

Cloud Messaging Protocol: AMQP vs HTTP

We saw a recent post from GitHub about removing the AMQP service from GitHub Services and passed it around the team as an item of interest. Got to talking later on that evening away from the keyboard and realized it has bigger meaning than just a side note.

Recent GitHub Blog Post

It’s not surprising that an AMQP binding might not be something to keep up for web services. The situation is similar to SOAP bindings in a sense. REST just started to take over, and the shift happened relatively organically. At some point with protocols, there’s a shift to an easier approach. Even a solid protocol will eventually get passed over for something lighter weight and more in keeping with current developer patterns (exceptions apply, especially if Microsoft is involved.)

That’s not unlike what’s happening here in our minds. AMQP is certainly not going anywhere. It’s a strong messaging standard that has a lot of backing and inertia behind it. Unfortunately, it’s built for a different time – one that is pre-cloud and behind the firewall.

Everyone Can Speak HTTP


One thing about AMQP that makes it difficult for cloud apps and web services (aside from its general complexity) is that it’s a separate application layer protocol than the one developers are used to using on a daily basis. Another key distinction is that it uses a less common default socket as part of the transport layer.

Certain cloud application hosts don’t allow most socket connections from within their sandbox but do allow HTTP requests. HTTP and HTTPS are always open on most enterprise firewalls, but special ports may not always be. Everyone can easily speak HTTP, but it takes special effort to speak AMQP. This greatly limits the environments into which AMQP can be deployed.

With something like the GitHub API services – where it can be a simple matter of posting an event to an API endpoint – the process should be as simple as possible and shouldn’t entail having to take a course in messaging protocols.

By using the same protocol the rest of the web relies on, REST and HTTP-based protocols gain all these benefits, plus the robust and well-tested optimizations, strategies, and tools that have been developed for the web.

Big Believers in HTTP as the Transport Protocol

Don’t get us wrong. AMQP is a comprehensive messaging protocol and its not going anywhere soon. We’re just not believers it’ll be the messaging protocol for the cloud. It’s still early in cloud messaging, but it’s becoming more clear what’s needed for simple, secure, and reliable messaging on a distributed, Internet-wide level.

It’s not a stretch to think the future of cloud messaging will include HTTP as the transport protocol, OAuth for authentication and security, and JSON as the primary message format.

From there, add in other web approaches and patterns and you get the messaging protocol that will be at the center of the next wave in connecting systems – although instead of just point-to-point or within a closed system, it’ll be on a globally connected scale.

Stay tuned.

Monday, February 25, 2013

Iron.io + OpenShift : PaaS 2.0 for the Enterprise


Iron.io is pleased to announce we are now a developer partner on OpenShift.

OpenShift is Red Hat's auto-scaling Platform as a Service (PaaS). Red Hat has a rich history when it comes to technology and large IT systems and, with OpenShift, they're on a serious path to make history happen again, this time in the world of cloud computing.


The pairing puts a rock-solid cloud platform together with Iron.io's high availability message queuing and task processing services. These capabilities are core needs within any production-scale cloud application. IronMQ, IronWorker, and IronCache are perfect fits for OpenShift applications that are looking for easy-to-use highly reliable services that can handle whatever capacity you want to send them.

The Next Step in Distributed Cloud Applications

A new era in PaaS is emerging. It's one that features platforms that support multiple languages and application frameworks, has solid security underpinnings, and is built to support both public and private installations. OpenShift is paving the way in this new era — one that is especially meant for organizations with a variety of distributed cloud applications and systems.

These distributed applications and systems will have many critical connections to other systems, both internally and with third-party services. Simple and reliable gateways and messages buses are needed to connect, coordinate, and process the message and event flow taking place between systems.

An application that can deploy and scale easily needs messaging and task processing that is also flexible, reliable, and easy to connect with. IronMQ, IronWorker, and IronCache provide this flexibility in concert with industrial-strength performance and capacity.

This pairing of Iron.io's services with OpenShift is not only the beginning of a solid relationship but also a glimpse into what the next era of PaaS looks like. Along the way, it's also a big win for organizations and cloud developers.

Getting Started with IronMQ on OpenShift

We've created an example on GitHub to help developers get started with IronMQ on OpenShift. It's a simple process with 4 steps:
  1. Create OpenShift App
  2. Configure IronMQ
  3. Deploy You App
  4. Profit! (actually it's View Your App but this way sounds better)
For the complete steps, check out the IronMQ on OpenShift repo and let us know what you think.

Thursday, February 21, 2013

Iron.io @ API Strategy Conference (NYC)


Iron.io will be in NYC this week at the API Strategy Conference. It was originally scheduled for November but Hurricane Sandy introduced a change in plans. It's being put on by Kin Lane (API Evangelist) and the people at 3Scale. It's on Thurs/Fri, Feb 21-22nd and is expected to draw a capacity crowd (site says it's sold out).

Iron.io @ API Strategy and Practice 2013

Here's the panel that Travis is on:
Track 3: API Security and ScalabilityAs APIs gain adoption they become ever more critical gateways to a company’s core business - ensuring access is secure and scalable are mission critical for your business. 
  • Paul Madsen (@PingIdentity) of  Ping Identity
  • Mark O'Neil (@TheMarkONeill) of Vordel
  • Travis Reeder (@treeder) of Iron.io
  • Andy Thurai (@AndyThurai) of Intel
Travis Reeder will be there with some t-shirts in hand. It you see him, hit him up for one. He might have a few "Callback Me Maybe" ones on hand but we're just about out of that version.

Monday, February 18, 2013

Using IronMQ as a Celery Broker

We are developers because we love to build stuff. Whether we're building an e-commerce site, a mobile game, or a network of connected devices, we love the thrill of solving problems and innovating. Modern frameworks that save us time in the plumbing allow us to focus on our own innovations – ultimately making us happier and more satisfied developers.

Celery is one of these frameworks. It is a popular Python-based distributed task queue for processing asynchronous and scheduled jobs – something that every application needs and every developer should understand. Behind Celery, you can choose one of the many popular queue technologies such as RabbitMQ for the transport. In this post, I'm going to show you how easy it is to use IronMQ as the Celery transport.


Why IronMQ

The reason for IronMQ is Simple: instant high availability, reliability, and scalability. IronMQ is a powerful elastic cloud service designed for the modern application stack. There is no installation, no maintenance, no upgrades to worry about, and a lot of 9's in our uptime (6 to be exact for December). RabbitMQ and Redis are good pieces of technology, but as many can attest, they leave much to be desired when it comes to scaling, high availability, and technical erosion.

In addition, you'll get great features only cloud services can add such as beautiful dashboards, reports, alerts, webhooks, continual smarter queue innovation, and more. All of this combined with a very large free tier makes IronMQ and Celery a very nice pairing.


3 Easy Steps to Setup IronMQ

Step 1

Install the iron-celery Python module from pip.

pip install iron_celery

Step 2

Import the module into your Celery application (line 2 below).


Step 3

Modify the broker to point to IronMQ (lines 4-5). Retrieve your project ID and token by signing up for a free account at Iron.io.



The Results Store

Many developers want to also keep track of task state and thus need to tell Celery to rely on a datastore. A queue is not the best place to handle this. That's where IronCache comes in. With one configuration change, you can tell Celery to track task state in IronCache so that in a single line of code you've eliminated the need to worry about both your broker and your datastore.

Simply pass in the "backend" variable to your Celery app (line 6):


Where to Go From Here

If you are already using Celery, great, trying out IronMQ is free and easy. Simply visit Iron.io/celery to setup your free account and get your IronMQ credentials.

If you are researching, we encourage you to visit CeleryProject.org and follow the initial tutorials to get started. Celery is a super powerful tool that will ultimately save you a lot of time and effort in the plumbing. Pair it with IronMQ for a highly available, scalable, and maintenance-free broker, and you'll be spending much more of that precious time coding.

Now go and build.


For more information, a short video tutorial, and to signup for a free account, visit Iron.io/celery



Tuesday, February 12, 2013

Game of Nines - Iron.io Uptime Report

The number one core tenet of Iron.io is IronClad Reliability so we put a lot of time and effort into making our services reliable. It seems to have paid off. Our uptime for IronMQ was 100% for the past 30 days and 99.98% for the past 6 months.

For comparison, "Gmail uptime is in the range of 99.99 percent – meaning the average user experiences about four minutes of downtime per month – and Amazon targets 99.95 percent for AWS." [source]

Note: some companies do not include scheduled maintenance in their uptime numbers so if they have scheduled downtime it doesn't count. We never do scheduled maintenance so these numbers are real.


Past 30 Days: 100% Uptime




IronMQ - Past 30 Days

Past 6 Months: 99.98% Uptime


IronMQ - 6 Months

Past 1 Year: 99.93% Uptime


IronMQ - Past year

Reliability and the Cloud

If you've been scared of using the cloud, we hope these numbers and the numbers of our peers will make you reconsider. There's a good article written by +Dave Girouard, former President of Enterprise for Google, called "The delusions that companies have about the cloud", that talks about some of the false fears of using the cloud.

While we strive for lots of 9's, the past does not represent the future so things could change, but we'll do our best to keep getting better. Knock on wood.

Monday, February 11, 2013

Improvement to HUD - How to Better Manage Your Tasks

We did a post the other day on improvements we made to the IronMQ HUD to make it easier to manage your message queues. We also made some changes to the HUD to make it easier to manage your tasks and scheduled jobs. Here's the major changes for the IronWorker HUD.


IronWorker - Task List

The task list got a facelift both in look and organization. We changed the icons to give you a clearer view of the status of tasks. We also organized the columns better and added a duration icon. Simple, clean, and, as Chad likes to say, beautiful.

View of Worker Tasks

IronWorker - Scheduled Tasks

For scheduled tasks, we organized things a bit better to draw out the scheduled tasks that ran and when they're supposed to run again. The goal is to provide you with a clear view at a glance of all the things that are happening on a schedule in your app.

View of Scheduled Tasks

Let Us Know Your Thoughts

If you have any special requests – something that will make our services better or easier to use – please let us know. Send us a note via our support channel or check in at our public chat room. The team loves the input.

Friday, February 1, 2013

Improvement to HUD - How to Better Manage Your Queues


If you're an Iron.io user, you’ve probably noticed improvements in our HUD over the last few months. We’ve gotten great input from customers on what they wanted to see and we took that to heart. We also released some new features like push queues and needed a way to bring them out within the dashboard.

Here’s a rundown of some of the things we’ve added to the HUD. We're confident users will find them useful in helping them manage their queues.

IronMQ - Messages vs Queue Size

We’ve improved the analytics screens for IronMQ so that you can see queue sizes against messages on an hourly basis.

For most applications, messages are the lifeblood of a working system. Seeing how your app is handling messages over time is critical. This view now gives you a key snapshot to stay on top of your system’s performance.

Messages vs Queue Size



IronMQ - Messages vs API Requests

We’ve also added an aggregate view of messages and API requests over time by project. Note that there’s no right metric here – some systems may poll more frequently than others. In these cases, the ratio between requests and messages may change over time.

With queues with less polling or that use push queues, the ratio of messages to requests may be more tightly aligned. (Note that even with frequent polling, the requests won’t come even close to the amount we provide free so you should poll away if you need to stay on top of inbound messages.)

Messages vs API Requests



IronMQ - Push Queue Control

Push Queues are a big feature that we released recently. Instead of polling for messages on the queue, push queues let developers send messages to subscribers whenever they hit the queue. This pattern provides greater flexibility when it comes to message handling. For one, you can eliminate a polling process and just send messages to a destination. For another, you can reduce the maximum time that messages might spend on the queue.

You can control push queues via the API but you can also do so now via the HUD. Want to adjust the retry limit or the delay between retries, or add an endpoint for message to hit? Not a problem. Just pull up the queue and adjust it in the HUD.

Push Queue Control