How to set up Celery and RabbitMQ

MAUS can be used with the Celery asynchronous distributed task queue ( to allow transform (map) steps to be executed on multiple processors in parallel.

Celery uses RabbitMQ ( as a broker to dispatch jobs to processing nodes.

Set up Celery

Celery is a Python tool which is automatically downloaded and installed when you build MAUS.

Set up RabbitMQ

To install on Scientific Linux or RedHat you should:

  • Log in as a super-user by using sudo su - or su.
  • Run:
    $ yum install rabbitmq-server
     rabbitmq-server        noarch        2.2.0-1.el5          epel           890 k
    Installing for dependencies:
     erlang                 i386          R12B-5.10.el5        epel            39 M
     unixODBC               i386          2.2.11-7.1           sl-base        832 k
  • Check that /usr/sbin is in the PATH:
    $ echo %PATH
  • If not, then add it:
    $ export PATH=$PATH:/usr/sbin
  • Start the RabbitMQ server:
    $ /sbin/service rabbitmq-server start
  • Create a MAUS username and password pair e.g.
    $ rabbitmqctl add_user maus suam
    Creating user "maus" ...
  • Create a MAUS virtual host:
    $ rabbitmqctl add_vhost maushost
    Creating vhost "maushost" ...
  • Set the permissions for the user on this host:
    $ rabbitmqctl set_permissions -p maushost maus ".*" ".*" ".*" 
    Setting permissions for user "maus" in vhost "maushost" ...
  • Check it is running OK:
    $ /sbin/service rabbitmq-server status
    Status of all running nodes...
    Node 'rabbit@maus' with Pid 1377: running

Default port:

By default RabbitMQ uses port 5672. If you want worker nodes outside your firewall to use the RabbitMQ broker then you will need to open this port.

For more information see:

Configure nodes as Celery workers

Ensure the MAUS software is deployed on the nodes you want to use as workers and that you have run

$ source

Within the MAUS software directory, edit src/common_py/mauscelery/ and change

BROKER_HOST = "localhost" 

to have the full hostname of the host on which RabbitMQ was deployed e.g.

Run a quick test

Start the Celery workers on each node
$ celeryd -l INFO --purge

Wait for the Celery workers on each node to start. This may take a minute or two.
  • You should get output ending like
    [2012-04-27 11:51:38,391: WARNING/MainProcess] celery@miceonrec01a has started.
  • If you get output like
    [2012-04-27 11:43:59,611: ERROR/MainProcess] Consumer: Connection Error: [Errno 111] Connection refused. Trying again in 4 seconds...
    [2012-04-27 11:44:03,611: ERROR/MainProcess] Consumer: Connection Error: [Errno 111] Connection refused. Trying again in 6 seconds...

    It means rabbitmq is not set up.

Now, on any node with MAUS deployed, run

$ ./bin/examples/ -type_of_dataflow=multi_process \ 

The client should show information on spills being passed to Celery and the results returned.

You can also run the MAUS Celery integration tests:

$ python tests/integration/test_distributed_processing/

A lot of messages will be printed. However, the run should end with:
Ran 11 tests in 76.748s


More information

See the MAUS pages on,

Updated by Rogers, Chris about 11 years ago ยท 18 revisions