Fastest Messaging Queue: IronMQ is 10x Faster Than RabbitMQ
We wrote IronMQ from the ground up as a cloud-agnostic message queue service with a focus on performance and easy deployment and management. Since its inception, we’ve put all our highest volume customers on it, some doing billions of message requests per day.
As our technology continues to evolve, it’s important that we continue to measure our success against industry benchmarks and the competition. A popular name in message queue services is RabbitMQ. So, are we faster than RabbitMQ?
Table of Contents
IronMQ vs RabbitMQ: The Setup
This comparison looked at IronMQ vs RabbitMQ in various message queue scenarios. These tests were all run on the same hardware: one box running IronMQ or RabbitMQ and another box on same EC2 VPC running the benchmark. These tests happened on AWS EC2 with c3.8xl instance types, which have the following specs:
- 32 vCPUs
- 2x200 GB SSDs
- 60 GB of RAM
- 10 GB connection
We’ve enabled disk persistence in all the tests unless stated otherwise, to ensure fairness and consistency, since IronMQ is always persistent. In-memory tests with RabbitMQ and other MQs might provide different performance numbers, but we feel strongly that message persistence and data integrity are paramount and so only offer IronMQ in persistent and clustered configurations.
We used the CoreOS AMI "CoreOS-stable-681.2.0-hvm”, IronMQ version 3.1.15 and RabbitMQ version 3.5.3. Both IronMQ and RabbitMQ were set up as a single node servers.
The benchmarks are open source at iron-maiden . Times are recorded in milliseconds and rates are per second.
Produce and Consume
The following tests produce messages as fast as possible and consume them as fast as possible, in various scenarios. These are similar to RabbitMQ's own benchmarks.
Test 1: 100 producers, 100 consumers, 1KB message size, 1 queue
As you can see, IronMQ can handle a sustained enqueue rate of ~18,000 - ~19,000 messages per second and a consume rate of ~4K per second on a single queue.
RabbitMQ can handle ~950 per second in and out.
Test 2: 100 producers, 100 consumers, 1KB message size, 10 queues
This one increases the number of queues which should reduce contention on the head of the queue during dequeue.
The enqueue rate didn't change much, but as you can see, the dequeue rate increased by 2.5x, from 4K per second to 10.5-11K per second. 
RabbitMQ didn't change much by adding more queues.
Test 3: 100 producers, 100 consumers, 1KB message size, 100 queues, 100 batch
In Test 3, we increased the batch size to post 100 messages per request vs 1 per request in the previous tests and also increased the number of queues to 100.
Getting close to 100K producing and 40K consuming on IronMQ.
15-16K producing and consuming on RabbitMQ.
Test 4: 100 producers, 100 consumers, 1 byte message size, 10 queues, 100 batch
All the previous tests were using 1KB messages, this one uses small single byte messages. Typical use would probably be somewhere in the middle. In cases where throughput matters, however, as in the case of high frequency messaging or IoT workloads, smaller message size should be a desired architectural characteristic.
Now we start to see a very desirable ~175K per second enqueue rate. Dequeue sits at around 20k. The more queues you use, the better the dequeue rate is with IronMQ.
In Test 4, RabbitMQ comes in around 22K per second in and out.
IronMQ > RabbitMQ
Speak to us to learn why RabbitMQ users are switching over to IronMQ.
Thanks for subscribing! Please check your email for further instructions.
Rate and Latency By Message Size
Next, message rate and latency of IronMQ vs RabbitMQ was measured by message size.
The next image shows average latency (lower is better):
Here is 99th percentile latency:
Fill and Drain
These tests produce and consume separately a set number of messages. In this case, we push 1 million messages then drain all of the messages.
IronMQ took 39.6 seconds to push 1 million messages onto a single queue, then 1 minutes and 45 seconds to drain them.
RabbitMQ without persistence took 7m28s to push 1 million messages and 7m59s to drain them.
RabbitMQ, with persistence on, took 9m13s to push 1 million messages the queue and 8m17s to drain them.
Bonus: XFS vs EXT4
As a bonus message queue test, we wanted to show you IronMQ running on two different file systems: Ext4 and XFS. XFS gave us at least 20% better performance overall. Gains were considerably more noticeable when using larger messages sizes. In the below graphs, each message was 32KB which shows a near doubling in performance near the end of the graph. There is still an inevitable message rate drop over time due to database compaction, however the decrease is a lot less using XFS.
And XFS:
Which Message Queue Service is Fastest?
It’s true that when it comes to message queue services, IronMQ and RabbitMQ are fairly different products with different goals. IronMQ’s goal is to be the best cloud-agnostic message queue on every cloud, including behind the firewall and even as an on-premise solution. IronMQ is a multi-tenant, highly available, horizontally scalable, HTTP based message queue. One IronMQ cluster for your organization can support any number of employees or projects – you don’t need one instance per project.
The results of our comparisons show that IronMQ is lightning fast, in many instances faster than RabbitMQ and many other competitors, giving IronMQ the edge in a fast-moving digital world.
Here's a quote that sums up the performance improvements in IronMQ, from one of our customers that switched from IronMQ v2 to v3.
Thanks to Brandon Gonzalez @bg451 for building and running all these benchmarks and creating all the pretty graphs used in this post.
Switch from RabbitMQ to IronMQ
A fast, reliable messaging queue is important for every application. Switch from RabbitMQ to IronMQ and get the speed you need in order to scale.