Push Queues + Error Queues = Better Queue Processing
Overview
IronMQ has had push queue capabilities for a while now. Push queues allow messages/payloads to be pushed to one or more endpoints and is a very popular feature. Within IronMQ push queues, up to 100 subscribers can be added – with the routing being either unicast (pushing to 1 subscriber) or multi-cast (pushing to all subscribers).
Push queues eliminate the need to poll as well as provide access to messages/payloads more promptly. (As soon as the message comes in, it gets distributed to one or more subscribers.)
To make push queues even more effective for transaction processing and other message handling needs, we recently added an error queue capability to IronMQ. If error queues are enabled, messages that fail – after a set number of retries – will get put onto an error queue. This makes it so you can easily recover if the subscribers are not available or under significant load.
 |
| Push Queues + Error Queues Improve Message Processing |
Table of Contents:
Achieve Cloud Elasticity with Iron
Speak to us to find how you can achieve cloud elasticity with a serverless messaging queue and background task solution with free handheld support.
Retries Help...
 |
| Retries help... |
With IronMQ, retries can be applied and so if a push fails, it’ll retry after a certain delay for up to n times (where n is set by the developer). A problem arises, however, if all the retries fails. You certainly don’t want lose messages and so with error queues, messages that fail on the push will be placed on an error queue.
... But Error Queues Pick Up after Retries Fail
Once a message hits an error queue, it can be dealt with as appropriate. The common approach with an error queue is to poll it on a periodic basis and run through a set of exception steps. This can include a combination of logging exceptions and possibly retrying (if the case of network outages or possibly api-related issues).
 |
| ...but errors still happen |
Error messaging can also include a completely different set of processing steps. Additionally, error queues could be turned into push queues but beware, because that can introduce the potential for dropping messages if pushes fail within that push queue.
Iron.io Serverless Tools
Speak to us to learn how IronWorker and IronMQ are essential products for your application to become cloud elastic.
Thanks for subscribing! Please check your email for further instructions.
Creating an Error Queue
Error queues are off by default. To create an error queue, simply post to your push queue a message with the error_queue option defined and a queue name provided.
{"push_type":"unicast", "subscribers": [ {"url": "http://thiswebsitewillthrowanerror.com"} ], "error_queue": "MY_EXAMPLE_ERROR_QUEUE"}
If a push queue is set with the error_queue parameter, then after the set number of retries, a message will be put in the named error queue and viewable via your account dashboard. By default, the error queue will be a pull queue.
NOTE: An error queue will not appear in your dashboard until an initial error message is received.
The Error queue message will contain the following information:
You can look up the original message if needed via the GET message endpoint using the source_msg_id value.
To turn off/disable an error queue, just post to your push queue set the error queue option to an empty string e.g. "error_queue": "".
{"push_type":"unicast", "subscribers": [ {"url": "http://thiswebsitewillthrowanerror.com"} ], "error_queue": ""}
NOTE: Omitting the “error_queue” option does not disable the error queue. You need to send it an empty string.
Push Queues + Error Queues = Better Processing
Adding error queues is a great feature that we're happy to have invested in. Now, you'll never have to worry about trying to find a message that went AWOL as it'll be right in your error queue. Let us know what you think.
Update: No post on errors and queues would be complete without a video of a falling penguin. (It’s better with the sound on.)
Unlock the Cloud with Iron.io
Find out how IronWorker and IronMQ can help your application obtain the cloud with fanatical customer support, reliable performance, and competitive pricing.
