Blog

Search This Blog

Loading...

Wednesday, July 24, 2013

IronMQ Fan-out Support

You can now add queues and workers as subscribers to your IronMQ queues so now a single message post can be pushed onto multiple queues and/or start IronWorker tasks. This is often referred to as "fan-out" and opens up a lot of interesting possibilities such as:

  • Multiple consumers of a single message. 
  • Better decoupling by adding queues as subscribers that different consumers can consume without the producer of the messages having to know about it. 
  • Better performance since you only need to push one message then let IronMQ deal with distributing it to multiple queues. 

Here's a simple diagram showing this pattern:


In this example, the queue called "Q1" has three other queues as subscribers, Q2, Q3, and Q4. Whenever a message is put on Q1, that message will be duplicated and pushed to all of it's subscribers which in this care are the three other queues. 

How To Add Queues and Workers as Subscribers


Using the new subscriber types is the same way you do with http subscribers, but the URL scheme is different. For instance, to post to an IronMQ queue, you can add a subscriber with a URL like:

    ironmq:///my_awesome_queue

This URL references a queue called "my_awesome_queue" and if this is one of your subscribers, a message put on the parent queue will be pushed to "my_awesome_queue" as well. 

And if you want an IronWorker task to start, you can use an IronWorker url like:

    ironworker:///my_awesome_worker

You pass these in to the queue update endpoint or the add/remove subscribers endpoints with json like the following:
That would set 5 subscribers on your queue, 3 subscribers are other queues, 1 is a worker and 1 is a normal http endpoint subscriber.

You can view the full API docs here. And you can read complete details about Push Queues here.

Conclusion

Being able to fan-out messages into other queues enables a lot of interesting messaging scenarios and we hope it can help solve some of your messaging problems. Give it a try and let us know what you think, we love feedback!