Message Queues for Buffering: An IronMQ and Python Case Study


Using IronMQ and Python to as a Buffer between Systems

Connecting systems and moderating data flows between them is not an easy task. Message queues are designed for just this purpose – buffering data between systems so that each can operate independently and asynchronously.

Here’s a short article on using IronMQ to buffer a CMS system from real estate data from a listing service. The app developer uses Python for the bridge code between the service and the CMS system.

Here’s an excerpt from the post:

Building a System with IronMQ and Python

One of my most recent projects was writing a system to deliver real estate listing data to a content management system. Since the listing data source was bursty and I wasn’t sure how the CMS would handle the load, I decided to use a message queue, where the messages would have a JSON payload. Message queues are great at decoupling components of a system.

For the queue, I used IronMQ. The company already was using it, it has a free tier (up to 24 messages a second), the service has been stable and reliable, it has great language SDKs, and setting up a durable message queue is something I’d rather outsource…

I wrote the bridge code from the listing database to the message queue in python. The shop was mostly Java and some Python, and Python seemed a better fit for a small ‘pull from here, push to there’ application…

[F]or this kind of problem, Python was a great solution. And I’ll reach for IronMQ any time I need a message queue. This pair of technologies was quick to implement, easy to deploy, and high performance wasn’t really a requirement, since the frequency of the listing delivery was the real bottleneck.

Read the full post >> 

About the Author

Dan Moore is a developer of web applications. Since 1999, Dan has created a variety of web sites and web applications, from ecommerce to portal to user self-service. He is competent in PHP, Java, perl, SQL, GWT, and object oriented design.


For other ways that you can use message queues, check out the following post from our archives.

Top 10 Uses For A Message Queue
We’ve been working with, building, and evangelising message queues for the last year, and it’s no secret that we think they’re awesome. We believe message queues are a vital component to any architecture or application, and here are ten reasons why:See more >>