Bug #1519
Method to get calibrations by run is broken
100%
Description
In the Calibrations class, the python API has a method to get detector calibrations by run
get_detector_calibration_for_run(device, run_number, calibration_type)
See http://heplnv156.pp.rl.ac.uk:4443/cdb/doc/python/classcdb_1_1__calibration_1_1Calibration.html
This is not working.
It does not respect the calibration_type argument and returns the same (possibly garbage) result regardless of
the requested type.
The other methods, get_current_calibration(device, type), and get_calibration_for_date(device, date, type) do indeed respond correctly
when the type or device or date are switched.
Attached python script to demonstrate and log
CDB version cdb-1.0.0.0
This is required to get calibrations by run for the TOF, and at some point for other detectors.
Files
Related issues
Updated by Martyniak, Janusz almost 9 years ago
Hi,
I looked in the server-side java code (the client-side Python looks healthy). It looks like the
public byte[] getDetectorCalibrationForRun(String device, String type, String runNo)
operation makes no use of either of the first 2 paramenters !
Looking at an implementation of
public String getCalibrationForRun(String name, String runNo)which looks sensible, obtaining correct data for a given run number consist of 2 steps:
- get a timestamp for the run
- get data for that timestamp using a ForDate(...) method.
Thus, it should not be complicated to fix, if I'm correct in my reverse-engineering attempt....and the ForDate(...) method is correct.
Durga, I'm unable to get one of your attachments (the log file). It might be useful to cross-check the implementation.
cheers, JM
Updated by Rajaram, Durga almost 9 years ago
- File get-cdb-calib-log.txt get-cdb-calib-log.txt added
Thanks, Janusz.
I've verified that get_current_calibration(..) and get_calibration_for_date(..) are indeed behaving correctly.
Have re-attached the log file as an ascii text
Also, the argument ordering is confusing -- the order seems different between the client side and the server side.
From the python API, cdb-1.0.0.0/cdb/_calibration.py:
line 120: def get_calibration_for_run(self, device, run_number, calibration_type=None)
line 142: _data = b64decode(self._server.getDetectorCalibrationForRun(device, calibration_type, run_number))
When you fix it, if it's not too much trouble, it may be good to keep the arg order the same to avoid headaches.
Updated by Martyniak, Janusz almost 9 years ago
Hi,
I uploaded the fix to the preprod server (preprodcdb.mice.rl.ac.uk). Durga, please have a look and run some tests if you can. If the fix is OK, I'll update a cdb.mice.rl.ac.uk as well.
all the best, JM
Updated by Rajaram, Durga almost 9 years ago
Hi Janusz -- the call is returning only hashes now. I see that the hashes are different, which is great, but I'd like to see the unencrypted output from the getters.
Thanks
Updated by Martyniak, Janusz almost 9 years ago
Dear All,
After having investigated the preprod DB, I found out that the DB instance contained wrong format binary data for Calibration, most likely due to a DB server upgrade for Postgres 8 to Postgres 9.
This made any tests of the calibration server side code useless, so I had no options but try to apply a fix to the production DB on heplnv156.Hers is the output from Python cmd line:from cdb import Calibration
import sys
_TEST_SERVER = 'http://cdb.mice.rl.ac.uk'
_DEVICE = 'TOF1'
_TYPE = 't0'
_RUN = 4000
_CALI = Calibration(_TEST_SERVER)
_RUN_CALI = _CALI.get_calibration_for_run(_DEVICE, _RUN, _TYPE)
_RUN_CALI
'TOFChannelKey 1 1 0 0 tof1 -76243.3 3\nTOFChannelKey 1 1 0 1 tof1 -47158.2 3\nTOFChannelKey 1 1 1 0 tof1 -76465.3 3\nTOFChannelKey 1 1 1 1 tof1 -48865.9 3\nTOFChannelKey 1 1 2 0 tof1 -76380.7 3\nTOFChannelKey 1 1 2 1 tof1 -48317.3 3\
(truncated)
So, as you can see the BLOB is decoded correctly.
Durga, feel free to test the service.
cheers JMUpdated by Rajaram, Durga almost 9 years ago
Hi Janusz -- I did a quick test earlier in the morning and it looked ok.
But by the time I got around to testing further, the CDB went down -- and is still down.
Updated by Martyniak, Janusz almost 9 years ago
Hi Durga,
I repeated your test right now and the CDB is OK. Have you tried your Python scripts and it looked down, or maybe you tried to visit http://cdb.mice.rl.ac.uk/cdb/ . This was another issue. I forgot to copy over an index.html file after installing new s/w. The CDB was in this state (missing front webpage, but services operational) for 2 or so hours after I applied the fix.
Otherwise the CDB looks healthy to me.cheers JMUpdated by Rogers, Chris almost 9 years ago
cdb went down with another OutOfMemory exception - I restarted it and set up a cronjob to do regular restarts on tomcat. It has restarted smoothly at 9:03, 10:03, 11:03, 12:03 without downtime so I have now moved the restart to a nightly at 03:03 am. I added documentation in the relevant pages of the service description. I was writing it up in the CDB issue tracker #1464
Updated by Rajaram, Durga almost 9 years ago
Checked for different devices and different runs and dates and Janusz, your fix works. The get_calibration_for_run does return the right objects now.
Thanks a lot!
Updated by Martyniak, Janusz almost 9 years ago
- Status changed from Open to Closed
- % Done changed from 0 to 100
As checks done by Durga confirm that the operation returns correct data I'm closing the bug.