Monday, February 24, 2014

Rabbit MQ - Replication Options

When you need to replicate data with Rabbit MQ there are three main choices:
  • Federation plugin (Federated Exchanges).
  • Shovel Plugin
  • Mirror Queues (Part of the basic Rabbit clustering)
A useful starting point when trying to decide which to use is to examine where the various options read messages from and where they write the messages to:

Message Source
Exchange Queue
Target Exchange Federated Exchange Shovel
Queue Mirror Queue

Note: The federation plugin actually creates a temporary queue bound to the source exchange in order to collect messages however since this queue is configured automatically you are effectively reading messages from the source exchange.

However there is a much easier way to choose the right option for you:

If you are replicating data between clusters you need either the federation plugin or the shovel plugin.
If you are simply want redundant copies of the data within the same cluster use mirror queues.

In practice the choice between the federation and shovel comes down to:
  • The federation plugin is much simpler to setup, however it only supports "pulling data".
  • Where as the shovel plugin has to be manually configured, but data can be pulled or pushed**. 
** Pushing data means that the system where the shovel plugin is installed is pushing data to the other clusters. Where as pulling means that the system where shovel is installed is pulling data from the remote cluster.

Therefore if your pulling data your better off with the federation plugin, however if you need to push it you will be forced to use shovel.