RabbitMQ Work Queue example

Based on these tutorials, this repo contains a MWE of a master that sends messages to various modules: a builder, an analyzer, and a reporter.

Those modules then send a response back to the master.

Modules use the Quartermaster synclib library to interact with RabbitMQ.

Demo setup

• A RabbitMQ instance
• One master Job issuing messages in all queues
• Three dummy module ReplicaSets containing Pods waiting for different kind of messages

Configuration

Set the RabbitMQ address for the master and for all modules.

Run the demo

1. First, build and push all the images:

   make images

2. Start RabbitMQ:

   kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.3/examples/celery-rabbitmq/rabbitmq-service.yaml
   kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.3/examples/celery-rabbitmq/rabbitmq-controller.yaml

3. Wait for RabbitMQ to start:

   kubectl logs $(kubectl get pod --selector component=rabbitmq -o=name) --follow

4. Start any subset of dummy modules:

   kubectl apply -k modules/builder
   kubectl apply -k modules/analyzer
   kubectl apply -k modules/reporter

   or, to start them all:

   kubectl apply -k modules

5. Start the master:

   kubectl apply -k master

Exactly one Pod of each module ReplicaSet will consume its corresponding message.
All the modules will then reply after having performed some dummy computation.