Using IronMQ as a Celery Broker
We are developers because we love to build stuff. Whether we’re building an e-commerce site, a mobile game, or a network of connected devices, we love the thrill of solving problems and innovating. Modern frameworks that save us time in the plumbing allow us to focus on our own innovations – ultimately making us happier and more satisfied developers.
Celery is one of these frameworks. It is a popular Python-based distributed task queue for processing asynchronous and scheduled jobs – something that every application needs and every developer should understand. Behind Celery, you can choose one of the many popular queue technologies such as RabbitMQ for the transport. In this post, I’m going to show you how easy it is to use IronMQ as the Celery transport.
Why IronMQ
The reason for IronMQ is Simple: instant high availability, reliability, and scalability. IronMQ is a powerful elastic cloud service designed for the modern application stack. There is no installation, no maintenance, no upgrades to worry about, and a lot of 9’s in our uptime (6 to be exact for December). RabbitMQ and Redis are good pieces of technology, but as many can attest, they leave much to be desired when it comes to scaling, high availability, and technical erosion.
In addition, you’ll get great features only cloud services can add such as beautiful dashboards, reports, alerts, webhooks, continual smarter queue innovation, and more. All of this combined with a very large free tier makes IronMQ and Celery a very nice pairing.
3 Easy Steps to Setup IronMQ
Step 1
Install the iron-celery Python module from pip.
pip install iron_celeryStep 2
Import the module into your Celery application (line 2 below).
Step 3
Modify the broker to point to IronMQ (lines 4-5). Retrieve your project ID and token by signing up for a free account at Iron.io.
The Results Store
Many developers want to also keep track of task state and thus need to tell Celery to rely on a datastore. A queue is not the best place to handle this. That’s where IronCache comes in. With one configuration change, you can tell Celery to track task state in IronCache so that in a single line of code you’ve eliminated the need to worry about both your broker and your datastore.
Simply pass in the “backend” variable to your Celery app (line 6):
Where to Go From Here
If you are already using Celery, great, trying out IronMQ is free and easy. Simply visit Iron.io/celery to setup your free account and get your IronMQ credentials.
If you are researching, we encourage you to visit CeleryProject.org and follow the initial tutorials to get started. Celery is a super powerful tool that will ultimately save you a lot of time and effort in the plumbing. Pair it with IronMQ for a highly available, scalable, and maintenance-free broker, and you’ll be spending much more of that precious time coding.
Now go and build.
For more information, a short video tutorial, and to signup for a free account, visit Iron.io/celery
