RabbitMQ Architecture
General purpose message broker—uses variations of request/reply, point to point, and
pub-sub communication patterns.
Smart broker / dumb consumer model—consistent delivery of messages to consumers, at
around the same speed as the broker monitors the consumer state.
Mature platform—well supported, available for Java, client libraries, .NET, Ruby, node.js.
Offers dozens of plugins.
Communication—can be synchronous or asynchronous.
Deployment scenarios—provides distributed deployment scenarios.
Multi-node cluster to cluster federation—does not rely on external services, however,
specific cluster formation plugins can use DNS, APIs, Consul, etc.
Apache Kafka Architecture
High volume publish-subscribe messages and streams platform—durable, fast and
scalable.
Durable message store—like a log, run in a server cluster, which keeps streams of records
in topics (categories).
Messages—made up of a value, a key and a timestamp.
Dumb broker / smart consumer model—does not try to track which messages are read by
consumers and only keeps unread messages. Kafka keeps all messages for a set period of
time.
Requires external services to run—in some cases Apache Zookeeper.
What are the Best Use Cases?
Apache Kafka Use Cases
Apache Kafka provides the broker itself and has been designed towards stream processing
scenarios. Recently, it has added Kafka Streams, a client library for building applications and
microservices. Apache Kafka supports use cases such as metrics, activity tracking, log aggregation,
stream processing, commit logs and event sourcing.
The following messaging scenarios are especially suited for Kafka:
Streams with complex routing, throughput of 100K/sec events or more, with “at least once”
partitioned ordering
Applications requiring a stream history, delivered in “at least once” partitioned ordering.
Clients can see a ‘replay’ of the event stream.
Event sourcing, modeling changes to a system as a sequence of events.
Stream processing data in multi-stage pipelines. The pipelines generate graphs of real-time
data flows.
RabbitMQ Use Cases
RabbitMQ can be used when web servers need to quickly respond to requests. This eliminates the
need to perform resource-intensive activities while the user waits for a result. RabbitMQ is also used
to convey a message to various recipients for consumption or to share loads between workers under
high load (20K+ messages/second).
Scenarios that RabbitMQ can be used for:
Applications that need to support legacy protocols, such as STOMP, MQTT, AMQP, 0-9-1.
Granular control over consistency/set of guarantees on a per-message basis
Complex routing to consumers
Applications that need a variety of publish/subscribe, point-to-point request/reply messaging
capabilities.
RabbitMQ in a nutshell
Who are the players:
1. Consumer
2. Publisher
3. Exchange
4. Route
Kafka
Who are the players
1. Consumer / Consumer groups
2. Producer
3. Kafka source connect
4. Kafka sink connect
5. Topic and topic partition
6. Kafka stream
7. Broker
8. Zookeeper
RabbitMQ – Robust Messaging for applications
Site: http://www.rabbitmq.com/
Implemenation: Erlang
Protocol: AMQP, STOMP
Platform: Windows, Linux, MAC OS
License: Apache
Apache ActiveMQ – Fastest MQ
Site: http://activemq.apache.org/
Implemenation: Java
Platform: OS-Independent
License: Apache
Protocol: Mutliple
Apache Qpid – AMQP Enterprise Messaging Implementation
Site: http://qpid.apache.org
Implemenation: C++, Java
Platform: OS-Independent
License: Apache
Protocol: AMQP
ZeroMQ – Messaging Library
Site:http://www.zeromq.org/
Implemenation: C++
Platform: Windows, Linux
License: LGPL
Protocol: AMQP
Apache Rocket MQ – Alibaba
Site:https://rocketmq.incubator.apache.org/
Implemenation: Java
Platform: OS-Independent
License: Apache
Protocol: JMS,MQTT
Apache Kafka
Site:http://kafka.apache.org/
Implemenation: Scala
Platform: OS-Independent
License: Apache
Protocol: ?
MSMQ – Microsoft Messaging Queue (not Open Source)
Site: https://msdn.microsoft.com/en-us/library/ms711472(v=vs.85).aspx
Implemenation: C++
Popular open (or closed) source solutions:
Qutrunk — software-as-a-service messaging platform
Advanced Message Queuing Protocol (AMQP)
Amazon Simple Queue Service
Apache ActiveMQ
Apache Qpid
Celery (software)
Gearman
IBM Integration Bus
IBM MQ
IronMQ
Java Message Service
MQTT
Message-oriented middleware, (category)
Microsoft Message Queuing (known colloquially as MSMQ)
Microsoft Azure, particularly Azure storage queues
and AppFabric Service Bus
NATS
Oracle Messaging Cloud Service
OpenEdge Sonic MQ
QDB queues with message replay feature
RabbitMQ
Redis
StormMQ, an example of a message queuing service
SnakeMQ
HornetQ
TIBCO Enterprise Message Service
Enduro/X Middleware platform
ZeroMQ
