Bug #1519

Method to get calibrations by run is broken

Added by Rajaram, Durga almost 10 years ago. Updated over 9 years ago.

Python API
Start date:
11 July 2014
Due date:
% Done:


Estimated time:


In the Calibrations class, the python API has a method to get detector calibrations by run

get_detector_calibration_for_run(device, run_number, calibration_type)


This is not working.
It does not respect the calibration_type argument and returns the same (possibly garbage) result regardless of
the requested type.

The other methods, get_current_calibration(device, type), and get_calibration_for_date(device, date, type) do indeed respond correctly
when the type or device or date are switched.

Attached python script to demonstrate and log

CDB version cdb-

This is required to get calibrations by run for the TOF, and at some point for other detectors.

Files (2.13 KB) Rajaram, Durga, 11 July 2014 05:25
get-cdb-calib.log (48.1 KB) get-cdb-calib.log Rajaram, Durga, 11 July 2014 05:25
get-cdb-calib-log.txt (48.1 KB) get-cdb-calib-log.txt Rajaram, Durga, 11 July 2014 16:02

Related issues

Related to Configuration Database - Feature #993: detector calibrationOpenWilson, Antony23 May 2012

Related to Configuration Database - Bug #1464: CDB down againOpenMartyniak, Janusz09 May 2014


Updated by Martyniak, Janusz almost 10 years ago

I looked in the server-side java code (the client-side Python looks healthy). It looks like the

public byte[] getDetectorCalibrationForRun(String device, String type, String runNo)

operation makes no use of either of the first 2 paramenters !

Looking at an implementation of

public String getCalibrationForRun(String name, String runNo) 

which looks sensible, obtaining correct data for a given run number consist of 2 steps:
  1. get a timestamp for the run
  2. get data for that timestamp using a ForDate(...) method.

Thus, it should not be complicated to fix, if I'm correct in my reverse-engineering attempt....and the ForDate(...) method is correct.

Durga, I'm unable to get one of your attachments (the log file). It might be useful to cross-check the implementation.

cheers, JM


Updated by Rajaram, Durga almost 10 years ago

Thanks, Janusz.
I've verified that get_current_calibration(..) and get_calibration_for_date(..) are indeed behaving correctly.
Have re-attached the log file as an ascii text

Also, the argument ordering is confusing -- the order seems different between the client side and the server side.
From the python API, cdb-

line 120: def get_calibration_for_run(self, device, run_number, calibration_type=None)

line 142: _data = b64decode(self._server.getDetectorCalibrationForRun(device, calibration_type, run_number))

When you fix it, if it's not too much trouble, it may be good to keep the arg order the same to avoid headaches.


Updated by Martyniak, Janusz almost 10 years ago

I uploaded the fix to the preprod server ( Durga, please have a look and run some tests if you can. If the fix is OK, I'll update a as well.
all the best, JM


Updated by Rajaram, Durga almost 10 years ago

Hi Janusz -- the call is returning only hashes now. I see that the hashes are different, which is great, but I'd like to see the unencrypted output from the getters.


Updated by Martyniak, Janusz almost 10 years ago

Dear All,
After having investigated the preprod DB, I found out that the DB instance contained wrong format binary data for Calibration, most likely due to a DB server upgrade for Postgres 8 to Postgres 9.

This made any tests of the calibration server side code useless, so I had no options but try to apply a fix to the production DB on heplnv156.
Hers is the output from Python cmd line:

from cdb import Calibration
import sys
_TYPE = 't0'
_RUN = 4000
_CALI = Calibration(_TEST_SERVER)
_RUN_CALI = _CALI.get_calibration_for_run(_DEVICE, _RUN, _TYPE)

'TOFChannelKey 1 1 0 0 tof1 -76243.3 3\nTOFChannelKey 1 1 0 1 tof1 -47158.2 3\nTOFChannelKey 1 1 1 0 tof1 -76465.3 3\nTOFChannelKey 1 1 1 1 tof1 -48865.9 3\nTOFChannelKey 1 1 2 0 tof1 -76380.7 3\nTOFChannelKey 1 1 2 1 tof1 -48317.3 3\


So, as you can see the BLOB is decoded correctly.

Durga, feel free to test the service.

cheers JM

Updated by Rajaram, Durga almost 10 years ago

Hi Janusz -- I did a quick test earlier in the morning and it looked ok.
But by the time I got around to testing further, the CDB went down -- and is still down.


Updated by Martyniak, Janusz almost 10 years ago

Hi Durga,
I repeated your test right now and the CDB is OK. Have you tried your Python scripts and it looked down, or maybe you tried to visit . This was another issue. I forgot to copy over an index.html file after installing new s/w. The CDB was in this state (missing front webpage, but services operational) for 2 or so hours after I applied the fix.

Otherwise the CDB looks healthy to me.
cheers JM

Updated by Rogers, Chris almost 10 years ago

cdb went down with another OutOfMemory exception - I restarted it and set up a cronjob to do regular restarts on tomcat. It has restarted smoothly at 9:03, 10:03, 11:03, 12:03 without downtime so I have now moved the restart to a nightly at 03:03 am. I added documentation in the relevant pages of the service description. I was writing it up in the CDB issue tracker #1464


Updated by Rajaram, Durga almost 10 years ago

Checked for different devices and different runs and dates and Janusz, your fix works. The get_calibration_for_run does return the right objects now.
Thanks a lot!


Updated by Martyniak, Janusz over 9 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

As checks done by Durga confirm that the operation returns correct data I'm closing the bug.

Also available in: Atom PDF