Bug #1519

Method to get calibrations by run is broken

Added by Rajaram, Durga over 7 years ago. Updated over 7 years ago.

Python API
Start date:
11 July 2014
Due date:
% Done:


Estimated time:


In the Calibrations class, the python API has a method to get detector calibrations by run

get_detector_calibration_for_run(device, run_number, calibration_type)


This is not working.
It does not respect the calibration_type argument and returns the same (possibly garbage) result regardless of
the requested type.

The other methods, get_current_calibration(device, type), and get_calibration_for_date(device, date, type) do indeed respond correctly
when the type or device or date are switched.

Attached python script to demonstrate and log

CDB version cdb-

This is required to get calibrations by run for the TOF, and at some point for other detectors.

Files (2.13 KB) Rajaram, Durga, 11 July 2014 05:25
get-cdb-calib.log (48.1 KB) get-cdb-calib.log Rajaram, Durga, 11 July 2014 05:25
get-cdb-calib-log.txt (48.1 KB) get-cdb-calib-log.txt Rajaram, Durga, 11 July 2014 16:02

Related issues

Related to Configuration Database - Feature #993: detector calibrationOpenWilson, Antony23 May 2012

Related to Configuration Database - Bug #1464: CDB down againOpenMartyniak, Janusz09 May 2014


Updated by Martyniak, Janusz over 7 years ago

I looked in the server-side java code (the client-side Python looks healthy). It looks like the

public byte[] getDetectorCalibrationForRun(String device, String type, String runNo)

operation makes no use of either of the first 2 paramenters !

Looking at an implementation of

public String getCalibrationForRun(String name, String runNo) 

which looks sensible, obtaining correct data for a given run number consist of 2 steps:
  1. get a timestamp for the run
  2. get data for that timestamp using a ForDate(...) method.

Thus, it should not be complicated to fix, if I'm correct in my reverse-engineering attempt....and the ForDate(...) method is correct.

Durga, I'm unable to get one of your attachments (the log file). It might be useful to cross-check the implementation.

cheers, JM


Updated by Rajaram, Durga over 7 years ago

Thanks, Janusz.
I've verified that get_current_calibration(..) and get_calibration_for_date(..) are indeed behaving correctly.
Have re-attached the log file as an ascii text

Also, the argument ordering is confusing -- the order seems different between the client side and the server side.
From the python API, cdb-

line 120: def get_calibration_for_run(self, device, run_number, calibration_type=None)

line 142: _data = b64decode(self._server.getDetectorCalibrationForRun(device, calibration_type, run_number))

When you fix it, if it's not too much trouble, it may be good to keep the arg order the same to avoid headaches.


Updated by Martyniak, Janusz over 7 years ago

I uploaded the fix to the preprod server ( Durga, please have a look and run some tests if you can. If the fix is OK, I'll update a as well.
all the best, JM


Updated by Rajaram, Durga over 7 years ago

Hi Janusz -- the call is returning only hashes now. I see that the hashes are different, which is great, but I'd like to see the unencrypted output from the getters.


Updated by Martyniak, Janusz over 7 years ago

Dear All,
After having investigated the preprod DB, I found out that the DB instance contained wrong format binary data for Calibration, most likely due to a DB server upgrade for Postgres 8 to Postgres 9.

This made any tests of the calibration server side code useless, so I had no options but try to apply a fix to the production DB on heplnv156.
Hers is the output from Python cmd line:

from cdb import Calibration
import sys
_TYPE = 't0'
_RUN = 4000
_CALI = Calibration(_TEST_SERVER)
_RUN_CALI = _CALI.get_calibration_for_run(_DEVICE, _RUN, _TYPE)

'TOFChannelKey 1 1 0 0 tof1 -76243.3 3\nTOFChannelKey 1 1 0 1 tof1 -47158.2 3\nTOFChannelKey 1 1 1 0 tof1 -76465.3 3\nTOFChannelKey 1 1 1 1 tof1 -48865.9 3\nTOFChannelKey 1 1 2 0 tof1 -76380.7 3\nTOFChannelKey 1 1 2 1 tof1 -48317.3 3\


So, as you can see the BLOB is decoded correctly.

Durga, feel free to test the service.

cheers JM

Updated by Rajaram, Durga over 7 years ago

Hi Janusz -- I did a quick test earlier in the morning and it looked ok.
But by the time I got around to testing further, the CDB went down -- and is still down.


Updated by Martyniak, Janusz over 7 years ago

Hi Durga,
I repeated your test right now and the CDB is OK. Have you tried your Python scripts and it looked down, or maybe you tried to visit . This was another issue. I forgot to copy over an index.html file after installing new s/w. The CDB was in this state (missing front webpage, but services operational) for 2 or so hours after I applied the fix.

Otherwise the CDB looks healthy to me.
cheers JM

Updated by Rogers, Chris over 7 years ago

cdb went down with another OutOfMemory exception - I restarted it and set up a cronjob to do regular restarts on tomcat. It has restarted smoothly at 9:03, 10:03, 11:03, 12:03 without downtime so I have now moved the restart to a nightly at 03:03 am. I added documentation in the relevant pages of the service description. I was writing it up in the CDB issue tracker #1464


Updated by Rajaram, Durga over 7 years ago

Checked for different devices and different runs and dates and Janusz, your fix works. The get_calibration_for_run does return the right objects now.
Thanks a lot!


Updated by Martyniak, Janusz over 7 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

As checks done by Durga confirm that the operation returns correct data I'm closing the bug.

Also available in: Atom PDF