Monday, April 6, 2015

Comparison of Message broker (rabbitMq, ActiveMq,Kafka, ZeroMq)

Recently i got an opportunity to compare  Message brokers(rabbitMq, ActiveMq,Kafka, ZeroMq) for a specific business need. To do that i have selected few parameters which are  important for choosing message broker for important business functions. Overall i found that rabbitMq has features which are easy to configure, a very good admin console, very good support for development (almost in all languages), basic features of reliability available out of box(acknowledgement, confirmation, durability). We also did benchmarking and found publishing and consumption rate good. Detail comparison is as below:


ParameterRabbitMqActiveMqZero MqKafka
1. Clustering/Load Balancing mechanism.Clustering Available, Queues clustering have to be handled separately.Clustering queue will be only for HA not for load balancing FeatureAvailableCan be achieved by wriritng lots of customize code.Available but producer has to know to which partition it is writing..
2. Replication among different nodes.AvailableAvailableNot automatic as there is no broker but can be coded. But lot of customization.Available
3. Fault tolerance feature. Turned around time in case of failure.Durable Queue, Durable Message and Clustering support. Another cluster node will take over but in case of queue it is different(connection has to be established with new node again by client.)Durable Queue, topic and durable consumer supports and availability through clustering is ensured.Features available but not out of the box.Zookeeper is required to manage it.
4. Supported libraries for go and other languages like dot net (CRM , ERP and CMS are on window stack).Available in languages Java, Go, Python and .NetGo client not available. Rest based http interface is available.Go support availableAvailable support for Go.
5. securityBasic Level of Authentication like restricting users for read/write/configure(administration) exist.Authentication support using different plugin.One has to built on top of it.Not available in current version.
6. Interopretability in case Message broker is to be changed. (No binding)AMQP 0.9 complaint. So changing one AMQP complaint broker with another one should not need a change in client code. Rest based plugin available.Same as rabbitMq.It is AQMP 1.0 compliantSpecific client has to be written.Rest interface plugis are available.
7. Performance throughput (read/write).Moderate as per benchmarking data available. (I read in pivotal blog that it can receive and deliver more than one million messages per second.)Comparable to RabbitMq.Very fastVery fast
8. Administration interfaceAvailable, Http based having basic functionality.Basic Web console.Not available has to be built in.Very basic interface. Third party web console is available.Less features as compared to RabbitMq interface like User Management
9. Open SourceYesYesYesYes
10. Support for Big DataPublishing & Consumption rate comparison to kafka is less. So Can be a bottleneck in a situation like click straem where continuous publishing is required without a pause. One apache project "Flume" which can be used to tranfer data to Hadoop.Same as for rabbit MQ. Flume can also be used in active mq as it works with AMQP.Good in terms of fast writing and readingKafka Hadoop Consumer API
11. Push NotificationLibraries support both push and pull notification.Libraries support both push and pull notification.Libraries support both push and pull notification.Libraries support both push and pull notification.
12. OtherWorker has to manage what it has consumed or not. Broker does not take care of it. Message remain in the storage until a specified time. Worker has to provide partition id broker details.