Support #1299
Failed to upload geometry to configuration database
100%
Description
We are trying to upload a geometry to the configuration database. As a starting point we decided to upload to either rgma (old test CDB server) or preprod, as it is clear we don't know what we are doing. I should say we tried several permutations on the URI without success. Email trail below:
Hi Chris, So you know and before I forget, this was my last communication with Antony prior to leaving. Also, I have the latest version of the geometry uploaded to ".maus_release/files/geometry/upload". Executing the "upload_geometry" command that we were running will post the files in that directory to the CDB. Probably an unnecessary point but I thought I should tell you that the files are there and that they are legitimate. Cheers, Ryan Sent from my iPhone Begin forwarded message: From: "antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>" <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>> Date: 25 June, 2013 11:34:39 AM GMT+01:00 To: Ryan Bayes <Ryan.Bayes@glasgow.ac.uk<mailto:Ryan.Bayes@glasgow.ac.uk>> Cc: "Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>" <Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>> Subject: RE: issue with geometry upload from miceonrec01 you can use http://preprodcdb.mice.rl.ac.uk:8080 NB at some point this will switch to port 80 This is a new machine and I'm not sure if it is visible externally yet, the plan is to open port 80 to the outside world Regards Antony ________________________________ From: Ryan Bayes [Ryan.Bayes@glasgow.ac.uk<mailto:Ryan.Bayes@glasgow.ac.uk>] Sent: 25 June 2013 11:25 To: Wilson, Antony (STFC,RAL,SC) Cc: Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk> Subject: Re: issue with geometry upload from miceonrec01 Thank you. That is helpful. Another question: Mat suggested using the test server "rgma19.pp.rl.ac.uk<http://pp.rl.ac.uk>". Do you know if this server is indeed in operation. Thanks again, Ryan On 2013-06-25, at 10:29 AM, <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk><mailto:antony.wilson@stfc.ac.uk>> wrote: cdb.mice.ac.uk<http://cdb.mice.ac.uk><http://cdb.mice.ac.uk><http://cdb.mice.ac.uk> is the default server, not the default upload server, by default you get read only access. Matt has a machine in the control room that he uses for upload. It should be configured to use configdb.micenet.rl.ac.uk Regards Antony > From: Ryan Bayes [Ryan.Bayes@glasgow.ac.uk] > Sent: 25 June 2013 10:19 > To: Wilson, Antony (STFC,RAL,SC) > Cc: Matthew.Littlefield@brunel.ac.uk > Subject: Re: issue with geometry upload from miceonrec01 > > But I am now confused. > > I was trying from the mice control room. I was under the impression that would put me behind the firewall. > > Also, cdb.mice.ac.uk<http://cdb.mice.ac.uk> is the default upload server in the CDB software as distributed. I learned from Mat that there is a way to get around this but the correct server name is required, and I am not sure what that is. > > Respectfully, > Ryan > > > > On 2013-06-24, at 8:21 AM, <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>> <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>> wrote: > > Hi Ryan > > cdb.mice.rl.ac.uk provides read only access to the cdb. The cdb super mouse web service is running on a machine in the control room and has restricted access. If you need to upload a geometry to the production system then Mat should be able to help > > Regards > Antony > > > > -----Original Message----- > From: Ryan Bayes [mailto:Ryan.Bayes@glasgow.ac.uk] > Sent: 23 June 2013 17:55 > To: Wilson, Antony (STFC,RAL,SC); Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk> > Subject: issue with geometry upload from miceonrec01 > > Hello Antony, > > I know that you have moved on to bigger and brighter things, but I hoped you could help me with a problem or direct me to somebody else who could. > > I have been trying to upload a geometry to the MICE configuration data base, and I get the following feed back > > > [miceonrec01] utilities > ./upload_geometry.py --configuration_file=Config.py Traceback (most recent call last): > File "./upload_geometry.py", line 72, in <module> > main() > File "./upload_geometry.py", line 59, in main > (ul_dir, str(configuration.geometry_upload_note)) > File "/home/mice/MAUS/.maus_release/src/common_py/geometry/GDMLtoCDB.py", line 72, in __init__ > self.geometry_cdb = cdb.GeometrySuperMouse() > File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry_supermouse.py", line 40, in __init__ > "/cdb/geometrySuperMouse?wsdl") > File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry.py", line 44, in __init__ > super(Geometry, self).__init__(url, wsdl_dir) > File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_base.py", line 39, in __init__ > _cdb_wsdl) > cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at http://cdb.mice.rl.ac.uk/cdb/geometrySuperMouse?wsdl > > > The contents of the configuration file do not matter for the purpose of this problem since it seems to be in the ability of the program > to contact the server (which is defined in the SuperMouse method of CDB). Is there a different machine or user I should be doing this as? > Is there a different problem I should be aware of? > > Thank you, > Ryan Bayes > -- > Scanned by iCritical. > >
Updated by Rogers, Chris over 10 years ago
Few more comments. We are using MAUS 0.5.4, with one modification to GDMLtoCDB.py to stop cdb defaulting to URL from an environment variable; diff output:
=== modified file 'src/common_py/geometry/GDMLtoCDB.py' --- src/common_py/geometry/GDMLtoCDB.py 2013-02-13 14:49:22 +0000 +++ src/common_py/geometry/GDMLtoCDB.py 2013-06-25 09:57:43 +0000 @@ -68,8 +68,9 @@ generated automatically """ self.config = Configreader() - self.wsdlurl = None - self.geometry_cdb = cdb.GeometrySuperMouse() + self.wsdlurl = self.config.cdb_upload_url+\ + self.config.geometry_upload_wsdl + self.geometry_cdb = cdb.GeometrySuperMouse(self.wsdlurl) self.textfile = textfile self.text = "" if type(notes) != str:
working from onrec01. From MAUS command line we are running
python upload_geometry.py --configuration_file Config.py
where Config.py has following contents
simulation_geometry_filename = "/data/neutrino02/rbayes/MICE/devel_maus/maus/files/geometry/download/ParentGeometryFile.dat" accumulate_tracks = 1 geant4_visualisation = True verbose_level = 4 spill_generator_number_of_spills = 10 # simulation_geometry_filename = "files/geometry/download/RotatedGeometryFile.dat" # configuration database cdb_upload_url = "http://preprodcdb.mice.rl.ac.uk" # target URL for configuration database uploads TestServer::http://rgma19.pp.rl.ac.uk:8080/cdb/ # cdb_download_url = "http://rgma19.pp.rl.ac.uk:8080/cdb/" # target URL for configuration database downloads # geometry download geometry_download_wsdl = "geometry?wsdl" # name of the web service used for downloads geometry_download_directory = "%s/files/geometry/download" % os.environ.get("MAUS_ROOT_DIR") # name of the local directory where downloads will be placed geometry_download_by = 'current' # choose 'run_number' to download by run number, 'current' to use # the currently valid geometry or 'id' to use the cdb internal id # (e.g. if it is desired to access an old version of a particular # geometry) geometry_download_run_number = 0 geometry_download_id = 0 geometry_download_cleanup = True # set to True to clean up after download g4_step_max = 5.0 # this is the value which shall be placed in the Mice Modules which have been translated from CAD # geometry upload geometry_upload_wsdl = "" # "/cdb/geometrySuperMouse.wsdl" # name of the web service used for uploads geometry_upload_directory = "%s/files/geometry/upload" % os.environ.get("MAUS_ROOT_DIR") # name of the local directory where uploads are drawn from geometry_upload_note = "New step IV geometry includes tracker but not placement of tracker. Does include placement of TOFs, KL, Ckov, and EMR. Now uses the correct materials for beamline elements." # note, pushed to the server to describe the geometry. A note must be specified here (default will throw an exception). geometry_upload_valid_from = "2013-06-08 18:00:00" # date-time in format like: that the specified installation was made in the experiment. A date-time must be specified here (default will throw an exception). technical_drawing_name = "TD-1189-3004" #name and version of the technical drawing from which the CAD model came from. geometry_upload_cleanup = False # set to True to clean up after upload #dates need to get geomtry ids get_ids_start_time = "2012-01-01 09:00:00" get_ids_stop_time = "2013-10-11 09:00:00" get_ids_create_file = False #get beamline information # This executable will give the run numbers of the runs which the CDB has information on. # The information is the magnet currents, reasons for run and other information which # is specific to that run. When downloading a geometry by run number the beamline # information is merged with the geometrical infomation. Options for querying # beamline information are; 'all_entries' returns a list of all run numbers with beamline info. # 'run_number' prints whether info is held for this run number or not. # 'dates' returns a list of run numbers with info during specified time period. get_beamline_by = "dates" get_beamline_run_number = "" get_beamline_start_time = "2009-10-15 09:00:00" get_beamline_stop_time = "2013-10-11 09:00:00"
Updated by Rogers, Chris over 10 years ago
Problem was that
- the constructor takes a url which overloads the environment variable/default and is the url of the server
- set_url(...) takes a url which is the url of the wsdl
We were calling both the constructor and set_url, with the same url. So always one or the other url would throw an exception, resulting in failure.
Propose set_url(...) should be deprecated and constructor should be the way to do it.
Updated by Rogers, Chris over 10 years ago
Still having trouble - I do
python upload_geometry.py --configuration_file Config.py "http://configdba.micenet.rl.ac.uk" Traceback (most recent call last): File "upload_geometry.py", line 72, in <module> main() File "upload_geometry.py", line 59, in main (ul_dir, str(configuration.geometry_upload_note)) File "/home/mice/MAUS/.maus_release/src/common_py/geometry/GDMLtoCDB.py", line 74, in __init__ self.geometry_cdb = cdb.GeometrySuperMouse(self.wsdlurl) File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry_supermouse.py", line 40, in __init__ "/cdb/geometrySuperMouse?wsdl") File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry.py", line 44, in __init__ super(Geometry, self).__init__(url, wsdl_dir) File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_base.py", line 39, in __init__ _cdb_wsdl) cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at http://configdba.micenet.rl.ac.uk/cdb/geometrySuperMouse?wsdl
nmap responds with
nmap configdba -P0 -p80 Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2013-06-27 11:17 BST Interesting ports on configdba.micenet.rl.ac.uk (172.16.246.25): PORT STATE SERVICE 80/tcp filtered http Nmap finished: 1 IP address (1 host up) scanned in 0.006 seconds
What am I doing wrong now?
Updated by Wilson, Antony over 10 years ago
filtered = behind firewall
configdba.micenet.rl.ac.uk is listening on port 8080. I've just updated iptables so you can now talk to it on port 80 as well
Updated by Rogers, Chris over 10 years ago
You're right, I didn't spot it - but I didn't see anywhere in config db area or online area a requirement to talk on this port or that port... (or even what URL to use)
Updated by Rogers, Chris over 10 years ago
ps: upload completed now
Ryan, I made a new install of MAUS on onrec01 /home/mice/MAUS/.maus_cdb
for doing cdb uploads. Can you please use this and not the production install for doing cdb uploads.
Updated by Rogers, Chris over 10 years ago
- Status changed from Open to Closed
- % Done changed from 0 to 100
Closed the issue, Antony may want to raise something under CDB...