RabbitMQ configuration and monitoring¶
- Table of contents
- RabbitMQ configuration and monitoring
Celery uses RabbitMQ (http://www.rabbitmq.com/) as a broker to dispatch jobs to processing nodes.
All commands need to be run as a super-user by using sudo su -
or su
.
Celery connection information¶
To connect with RabbitMQ, Celery workers nespecify:
- Host name - name of host on which RabbitMQ is running. If local this can be
localhost
. - Port - by default RabbitMQ uses
5672
. If you want worker nodes outside your firewall to use the RabbitMQ broker then you will need to open this port. - Virtual host name -
maushost
- Username -
maus
- Password -
saum
Running the server¶
Start server¶
$ /sbin/service rabbitmq-server start Starting rabbitmq-server: SUCCESS rabbitmq-server.
Restart server¶
$ /sbin/service rabbitmq-server restart Restarting rabbitmq-server: SUCCESS rabbitmq-server.
Stop server¶
$ /sbin/service rabbitmq-server stop Stopping rabbitmq-server: rabbitmq-server
View server status¶
$ /sbin/service rabbitmq-server status Status of all running nodes... Node 'rabbit@maus' with Pid 24899: running done
Note that maus
is the name of the host on which RabbitMQ is running.
Server monitoring¶
rabbitmqctl
is a super-user command that allows current message queues and connections to RabbitMQ to be viewed. For full information, see http://www.rabbitmq.com/man/rabbitmqctl.1.man.html.
List the current connections to RabbitMQ¶
You can show the Celery workers that have connected. This can be useful for checking that a Celery worker has successfully connected. By default this shows the IP address and connection port of the Celery worker and the state of the connection.
$ rabbitmqctl list_connections Listing connections ... maus 129.215.62.184 51465 running maus 127.0.0.1 54750 running ...done.
The above shows two connections, one from a Celery worker running on the same host as RabbitMQ (address 127.0.0.1) and one running on a remote host (129.215.62.184).
You can request more connection information be shown e.g.
pid
- RabbitMQ connection ID.address
- RabbitMQ address used by worker to connect.port
- RabbitMQ port used by worker to connect.peer_address
- address of worker.peer_port
- port of worker.user
- RabbitMQ username used by worker to connect.vhost
- RabbitMQ virtual host used by worker to connect.state
- current state of connection.
$ rabbitmqctl list_connections pid address port peer_address peer_port user vhost state Listing connections ... <rabbit@maus.1798.0> 129.215.62.218 5672 129.215.62.184 51465 maus maushost running <rabbit@maus.1547.0> 127.0.0.1 5672 127.0.0.1 54750 maus maushost running ...done.
Both are connected to the MAUS virtual host, mausvhost
, with username maus
.
Browse the message queues¶
You can also browse the mesage queues. For these operations, rabbitmqctl
takes a -p
flag with the name of a virtual host (for MAUS this is typically called maushost
).
List queue names and sum of messages ready to be delivered and unacknowledged messages:
$ rabbitmqctl list_queues -p maushost Listing queues ... celery 0 worker2.celeryd.pidbox 0 worker1.celeryd.pidbox 0 ...done.
The above corresponds to two Celery workers having connected to RabbitMQ.
List queue names, number of messages ready to be delivered, number of unacknowledged messages and sum of these:
$ rabbitmqctl list_queues -p maushost name messages_ready messages_unacknowledged messages Listing queues ... celery 0 0 0 worker2.celeryd.pidbox 0 0 0 worker1.celeryd.pidbox 0 0 0 ...done.
The celery
queue is of interest as its messages_ready
count corresponds to the number of requests from a client that are awaiting to be delivered to a Celery worker node. This can be useful to check if a client's requests are, at least, getting to RabbitMQ successfully. For example, after a client invokes 8 Celery worker tasks:
celery 8 0 8 worker1.celeryd.pidbox 0 0 0 worker2.celeryd.pidbox 0 0 0 cffbe4e5989a48c8aed2411ceb64dd70 0 0 0 b9714b4775ca4985b1bc8d9a8dd64d9b 0 0 0 70f8edfcf33949bab255fbd4b02c2220 0 0 0 ee240f398517497e8b1036ffb96756e5 0 0 0 bfba8aa743f746849e5e97ed5855f056 0 0 0 1fdace10ae8f452abb0d676a4cb28c28 0 0 0 aa95178d13dd4168a32ce539a7368158 0 0 0 147c362d2496497ba7990cf07a5f1e27 0 0 0
You will also see, as above, one queue entry per task submitted. These have auto-generated names. As the tasks are executed and results await collection, the states of these will change. The celery
queue empties as the Celery worker's receive the entries. The auto-generated queues fill with the results, awaiting for collection by the clients.
celery 0 0 0 worker1.celeryd.pidbox 0 0 0 worker2.celeryd.pidbox 0 0 0 cffbe4e5989a48c8aed2411ceb64dd70 1 0 1 b9714b4775ca4985b1bc8d9a8dd64d9b 1 0 1 70f8edfcf33949bab255fbd4b02c2220 1 0 1 ee240f398517497e8b1036ffb96756e5 1 0 1 bfba8aa743f746849e5e97ed5855f056 1 0 1 1fdace10ae8f452abb0d676a4cb28c28 1 0 1 aa95178d13dd4168a32ce539a7368158 1 0 1 147c362d2496497ba7990cf07a5f1e27 1 0 1
And, after the client picks up the results,
celery 0 0 0 worker1.celeryd.pidbox 0 0 0 worker2.celeryd.pidbox 0 0 0 cffbe4e5989a48c8aed2411ceb64dd70 0 0 0 b9714b4775ca4985b1bc8d9a8dd64d9b 0 0 0 70f8edfcf33949bab255fbd4b02c2220 0 0 0 ee240f398517497e8b1036ffb96756e5 0 0 0 bfba8aa743f746849e5e97ed5855f056 0 0 0 1fdace10ae8f452abb0d676a4cb28c28 0 0 0 aa95178d13dd4168a32ce539a7368158 0 0 0 147c362d2496497ba7990cf07a5f1e27 0 0 0
Eventually, the task-specific queues will disappear.
Log files¶
RabbitMQ logs are in /var/log/rabbitmq/
.
Web management plug-in¶
RabbitMQ server 2.7+ supports a web-based management plug-in that allow connections, queues and a wealth of other information to be explored. For full information, see http://www.rabbitmq.com/management.html.
Installation¶
- Run:
$ rabbitmq-plugins enable rabbitmq_management The following plugins have been enabled: webmachine rabbitmq_mochiweb amqp_client rabbitmq_management_agent rabbitmq_management Plugin configuration has changed. Restart RabbitMQ for changes to take effect.
- Restart RabbitMQ:
$ /sbin/service rabbitmq-server restart Restarting rabbitmq-server: SUCCESS rabbitmq-server.
Use¶
- In a web browser, visit http://localhost:55672/mgmt/
- Login using username
guest
, passwordguest
.
Updated by Jackson, Mike about 11 years ago ยท 10 revisions