MOM Procedures: How to switch rack room elements off / on¶

List of Rack Room Elements¶

• Personnel Protection System (PPS)
  • This must not be turned off, and is not under MOM control. Contact an expert if required.
• Network
  • This must not be turned off, and is not under MOM control. Contact an expert if required.
• Infrastructure
• General MICE Servers
• Controls and Monitoring Systems
• DAQ
• Online Reconstruction Servers

Personnel Protection System (PPS)¶

This is not under MOM control. It should not be turned off by the MOM. If it is found to be off unexpectedly, contact an expert.

Network¶

This is not under MOM control. It should not be turned off by the MOM, unless arranged with the Online group with input from ISIS. In the event of the switch unstacking (losing connectivity between layers, see eLog 1606), contact RAL networking.

Infrastructure¶

The main elements are UPSs and Air Conditioning.

The A/C should be on when possible. If the A/C can not be on, then the DAQ crates will need to be switched off. Other elements may need to be switched off if the temperature increases, so communicate A/C downtime with the Online group.

The UPSs are obviously vital for the machines they power. Any problems with the UPSs should be communicated to the mice-dcm@mice.phys.iit.edu mailing list. UPSs should only be switched off if all of the connected elements are off. If it is not necessary to switch the UPS off, and it might be required soon, then leave it on so that it is charged when required.

General MICE Servers¶

These include:

• micestore
• miceserv1
• ConfigDB
• targetctl

These machines (especially miceserv1) are depended upon by many other systems. Don't shut them down unless you are sure that they are not required or that it is necessary. N.B. Some of the systems depending on these machines are not in the Rack Room, e.g. R9 Focus Coil tests.

Power Off Instructions¶

Each machine should be shutdown cleanly, from within Linux. Most can be accessed from one of the two user accounts on the DAQ KVM server. Shutdown in the following order:

1. targetctl
2. ConfigDB
3. micestore
4. miceserv1

Power On Instructions¶

Each machine should be monitored from the DAQ KVM system while booting, before moving to the next machine. This is especially important for miceserv1, which provides DHCP, DNS and NTP for all MICE Net computers. Start the machines in the following order:

1. miceserv1
2. micestore
3. ConfigDb
4. targetctl

Controls and Monitoring¶

The main C&M server is miceecserv, with miceecserv2 intended as a hot spare. Most of the IOCs are run from various VME crates (which probably don't need to be turned on/off) and miceiocpc1/miceiocpc2 for software IOCs. The IOC for the Decay Solenoid miceioc5 is not under MOM control, and should be left to the DS experts. The three OPI machines provide the main graphical interface to C&M.

Power Off Instructions¶

Each machine should be shutdown cleanly, from within Linux. All can be accessed via ssh, from miceecserv, which has a monitor/keyboard/mouse in the MLCR. Shutdown in the following order:

1. cagateway
2. miceisisgateway
3. micecss1
4. miceopipc1-3
5. miceiocpc1-2
6. miceecserv2
7. miceecserv

Power On Instructions¶

The order here is important. miceecserv2@, miceiocpc1, miceiocpc2 don't have monitors connected, so ensure that they are up via ssh.

1. miceecserv
• Ensure that this is up cleanly, and has internet access, before continuing.
1. miceecserv2
2. miceiocpc1
3. miceiocpc2
4. miceopipc1
• Once miceopipc1 is up, start the Server and Client menus from the desktop, and start the standard servers (Pierrick, I'd like to link to another page here, for a default state for the C&M).
1. miceopipc2
2. miceopipc3
3. miceisisgateway
4. cagateway

DAQ¶

These include:

• LDCs - miceacq06-10
• GDCs - miceraid4-5
• VME, NIM & CAMAC Crates (7 in total)
• LM

Power Off Instructions¶

Each machine should be shutdown cleanly, from within Linux. All can be accessed from one of the two user accounts on the DAQ KVM switch. Shutdown order is not important.

All of the crates have a physical power switch, 7 in total. Most are obvious, the only 'difficult' one is below the very dense cabling hidden behind the shield with the holes in it.

The power supplies for the Luminosity Monitor are facing backward in the C&M racks, opposite the DAQ crates. See manual .

Power On Instructions¶

Each machine should be monitored from the DAQ KVM system while booting, to ensure there are no errors. Start up order is not important.

The Crates and LM PSUs can be switched on in any order, as required.

Online Reconstruction¶

Instructions for the OnRec machines can be found at: [[computing-software:Onrec_power_cycle_procedure]].