RabbitMQ configuration and monitoring¶

• Table of contents
• RabbitMQ configuration and monitoring
  • Celery connection information
  • Running the server
    • Start server
    • Restart server
    • Stop server
    • View server status
  • Server monitoring
    • List the current connections to RabbitMQ
    • Browse the message queues
    • Log files
  • Web management plug-in
    • Installation
    • Use

Celery uses RabbitMQ (http://www.rabbitmq.com/) as a broker to dispatch jobs to processing nodes.

All commands need to be run as a super-user by using sudo su - or su.

Celery connection information¶

To connect with RabbitMQ, Celery workers nespecify:

• Host name - name of host on which RabbitMQ is running. If local this can be localhost.
• Port - by default RabbitMQ uses 5672. If you want worker nodes outside your firewall to use the RabbitMQ broker then you will need to open this port.
• Virtual host name - maushost
• Username - maus
• Password - saum

Running the server¶

Start server¶

$ /sbin/service rabbitmq-server start
Starting rabbitmq-server: SUCCESS
rabbitmq-server.

Restart server¶

$ /sbin/service rabbitmq-server restart
Restarting rabbitmq-server: SUCCESS
rabbitmq-server.

Stop server¶

$ /sbin/service rabbitmq-server stop
Stopping rabbitmq-server: rabbitmq-server

View server status¶

$ /sbin/service rabbitmq-server status
Status of all running nodes...
Node 'rabbit@maus' with Pid 24899: running
done

Note that maus is the name of the host on which RabbitMQ is running.

Server monitoring¶

rabbitmqctl is a super-user command that allows current message queues and connections to RabbitMQ to be viewed. For full information, see http://www.rabbitmq.com/man/rabbitmqctl.1.man.html.

List the current connections to RabbitMQ¶

You can show the Celery workers that have connected. This can be useful for checking that a Celery worker has successfully connected. By default this shows the IP address and connection port of the Celery worker and the state of the connection.

$ rabbitmqctl list_connections
Listing connections ...
maus    129.215.62.184  51465   running
maus    127.0.0.1       54750   running
...done.

The above shows two connections, one from a Celery worker running on the same host as RabbitMQ (address 127.0.0.1) and one running on a remote host (129.215.62.184).

You can request more connection information be shown e.g.

• pid - RabbitMQ connection ID.
• address - RabbitMQ address used by worker to connect.
• port - RabbitMQ port used by worker to connect.
• peer_address - address of worker.
• peer_port - port of worker.
• user - RabbitMQ username used by worker to connect.
• vhost - RabbitMQ virtual host used by worker to connect.
• state - current state of connection.

$ rabbitmqctl list_connections pid address port peer_address peer_port user vhost state
Listing connections ...
<rabbit@maus.1798.0>   129.215.62.218  5672    129.215.62.184  51465   maus   maushost running
<rabbit@maus.1547.0>    127.0.0.1       5672    127.0.0.1       54750   maus   maushost running
...done.

Both are connected to the MAUS virtual host, mausvhost, with username maus.

Browse the message queues¶

You can also browse the mesage queues. For these operations, rabbitmqctl takes a -p flag with the name of a virtual host (for MAUS this is typically called maushost).

List queue names and sum of messages ready to be delivered and unacknowledged messages:

$ rabbitmqctl list_queues -p maushost
Listing queues ...
celery  0
worker2.celeryd.pidbox  0
worker1.celeryd.pidbox  0
...done.

The above corresponds to two Celery workers having connected to RabbitMQ.

List queue names, number of messages ready to be delivered, number of unacknowledged messages and sum of these:

$ rabbitmqctl list_queues -p maushost name messages_ready messages_unacknowledged messages
Listing queues ...
celery  0       0       0
worker2.celeryd.pidbox  0       0       0
worker1.celeryd.pidbox  0       0       0
...done.

The celery queue is of interest as its messages_ready count corresponds to the number of requests from a client that are awaiting to be delivered to a Celery worker node. This can be useful to check if a client's requests are, at least, getting to RabbitMQ successfully. For example, after a client invokes 8 Celery worker tasks:

celery  8       0       8
worker1.celeryd.pidbox  0       0       0
worker2.celeryd.pidbox  0       0       0
cffbe4e5989a48c8aed2411ceb64dd70        0       0       0
b9714b4775ca4985b1bc8d9a8dd64d9b        0       0       0
70f8edfcf33949bab255fbd4b02c2220        0       0       0
ee240f398517497e8b1036ffb96756e5        0       0       0
bfba8aa743f746849e5e97ed5855f056        0       0       0
1fdace10ae8f452abb0d676a4cb28c28        0       0       0
aa95178d13dd4168a32ce539a7368158        0       0       0
147c362d2496497ba7990cf07a5f1e27        0       0       0

You will also see, as above, one queue entry per task submitted. These have auto-generated names. As the tasks are executed and results await collection, the states of these will change. The celery queue empties as the Celery worker's receive the entries. The auto-generated queues fill with the results, awaiting for collection by the clients.

celery  0       0       0
worker1.celeryd.pidbox  0       0       0
worker2.celeryd.pidbox  0       0       0
cffbe4e5989a48c8aed2411ceb64dd70        1       0       1
b9714b4775ca4985b1bc8d9a8dd64d9b        1       0       1
70f8edfcf33949bab255fbd4b02c2220        1       0       1
ee240f398517497e8b1036ffb96756e5        1       0       1
bfba8aa743f746849e5e97ed5855f056        1       0       1
1fdace10ae8f452abb0d676a4cb28c28        1       0       1
aa95178d13dd4168a32ce539a7368158        1       0       1
147c362d2496497ba7990cf07a5f1e27        1       0       1

And, after the client picks up the results,

celery  0       0       0
worker1.celeryd.pidbox  0       0       0
worker2.celeryd.pidbox  0       0       0
cffbe4e5989a48c8aed2411ceb64dd70        0       0       0
b9714b4775ca4985b1bc8d9a8dd64d9b        0       0       0
70f8edfcf33949bab255fbd4b02c2220        0       0       0
ee240f398517497e8b1036ffb96756e5        0       0       0
bfba8aa743f746849e5e97ed5855f056        0       0       0
1fdace10ae8f452abb0d676a4cb28c28        0       0       0
aa95178d13dd4168a32ce539a7368158        0       0       0
147c362d2496497ba7990cf07a5f1e27        0       0       0

Eventually, the task-specific queues will disappear.

Log files¶

RabbitMQ logs are in /var/log/rabbitmq/.

Web management plug-in¶

RabbitMQ server 2.7+ supports a web-based management plug-in that allow connections, queues and a wealth of other information to be explored. For full information, see http://www.rabbitmq.com/management.html.

Installation¶

• Run:
  

  $ rabbitmq-plugins enable rabbitmq_management
  
  The following plugins have been enabled:
    webmachine
    rabbitmq_mochiweb
    amqp_client
    rabbitmq_management_agent
    rabbitmq_management
  
  Plugin configuration has changed. Restart RabbitMQ for changes to take effect.
  

• Restart RabbitMQ:
  

  $ /sbin/service rabbitmq-server restart
  Restarting rabbitmq-server: SUCCESS
  rabbitmq-server.
  

Use¶

• In a web browser, visit http://localhost:55672/mgmt/
• Login using username guest, password guest.