Support #1299
Failed to upload geometry to configuration database
100%
Description
We are trying to upload a geometry to the configuration database. As a starting point we decided to upload to either rgma (old test CDB server) or preprod, as it is clear we don't know what we are doing. I should say we tried several permutations on the URI without success. Email trail below:
Hi Chris, So you know and before I forget, this was my last communication with Antony prior to leaving. Also, I have the latest version of the geometry uploaded to ".maus_release/files/geometry/upload". Executing the "upload_geometry" command that we were running will post the files in that directory to the CDB. Probably an unnecessary point but I thought I should tell you that the files are there and that they are legitimate. Cheers, Ryan Sent from my iPhone Begin forwarded message: From: "antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>" <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>> Date: 25 June, 2013 11:34:39 AM GMT+01:00 To: Ryan Bayes <Ryan.Bayes@glasgow.ac.uk<mailto:Ryan.Bayes@glasgow.ac.uk>> Cc: "Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>" <Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>> Subject: RE: issue with geometry upload from miceonrec01 you can use http://preprodcdb.mice.rl.ac.uk:8080 NB at some point this will switch to port 80 This is a new machine and I'm not sure if it is visible externally yet, the plan is to open port 80 to the outside world Regards Antony ________________________________ From: Ryan Bayes [Ryan.Bayes@glasgow.ac.uk<mailto:Ryan.Bayes@glasgow.ac.uk>] Sent: 25 June 2013 11:25 To: Wilson, Antony (STFC,RAL,SC) Cc: Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk> Subject: Re: issue with geometry upload from miceonrec01 Thank you. That is helpful. Another question: Mat suggested using the test server "rgma19.pp.rl.ac.uk<http://pp.rl.ac.uk>". Do you know if this server is indeed in operation. Thanks again, Ryan On 2013-06-25, at 10:29 AM, <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk><mailto:antony.wilson@stfc.ac.uk>> wrote: cdb.mice.ac.uk<http://cdb.mice.ac.uk><http://cdb.mice.ac.uk><http://cdb.mice.ac.uk> is the default server, not the default upload server, by default you get read only access. Matt has a machine in the control room that he uses for upload. It should be configured to use configdb.micenet.rl.ac.uk Regards Antony > From: Ryan Bayes [Ryan.Bayes@glasgow.ac.uk] > Sent: 25 June 2013 10:19 > To: Wilson, Antony (STFC,RAL,SC) > Cc: Matthew.Littlefield@brunel.ac.uk > Subject: Re: issue with geometry upload from miceonrec01 > > But I am now confused. > > I was trying from the mice control room. I was under the impression that would put me behind the firewall. > > Also, cdb.mice.ac.uk<http://cdb.mice.ac.uk> is the default upload server in the CDB software as distributed. I learned from Mat that there is a way to get around this but the correct server name is required, and I am not sure what that is. > > Respectfully, > Ryan > > > > On 2013-06-24, at 8:21 AM, <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>> <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>> wrote: > > Hi Ryan > > cdb.mice.rl.ac.uk provides read only access to the cdb. The cdb super mouse web service is running on a machine in the control room and has restricted access. If you need to upload a geometry to the production system then Mat should be able to help > > Regards > Antony > > > > -----Original Message----- > From: Ryan Bayes [mailto:Ryan.Bayes@glasgow.ac.uk] > Sent: 23 June 2013 17:55 > To: Wilson, Antony (STFC,RAL,SC); Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk> > Subject: issue with geometry upload from miceonrec01 > > Hello Antony, > > I know that you have moved on to bigger and brighter things, but I hoped you could help me with a problem or direct me to somebody else who could. > > I have been trying to upload a geometry to the MICE configuration data base, and I get the following feed back > > > [miceonrec01] utilities > ./upload_geometry.py --configuration_file=Config.py Traceback (most recent call last): > File "./upload_geometry.py", line 72, in <module> > main() > File "./upload_geometry.py", line 59, in main > (ul_dir, str(configuration.geometry_upload_note)) > File "/home/mice/MAUS/.maus_release/src/common_py/geometry/GDMLtoCDB.py", line 72, in __init__ > self.geometry_cdb = cdb.GeometrySuperMouse() > File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry_supermouse.py", line 40, in __init__ > "/cdb/geometrySuperMouse?wsdl") > File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry.py", line 44, in __init__ > super(Geometry, self).__init__(url, wsdl_dir) > File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_base.py", line 39, in __init__ > _cdb_wsdl) > cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at http://cdb.mice.rl.ac.uk/cdb/geometrySuperMouse?wsdl > > > The contents of the configuration file do not matter for the purpose of this problem since it seems to be in the ability of the program > to contact the server (which is defined in the SuperMouse method of CDB). Is there a different machine or user I should be doing this as? > Is there a different problem I should be aware of? > > Thank you, > Ryan Bayes > -- > Scanned by iCritical. > >
Updated by Rogers, Chris over 9 years ago
Few more comments. We are using MAUS 0.5.4, with one modification to GDMLtoCDB.py to stop cdb defaulting to URL from an environment variable; diff output:
=== modified file 'src/common_py/geometry/GDMLtoCDB.py' --- src/common_py/geometry/GDMLtoCDB.py 2013-02-13 14:49:22 +0000 +++ src/common_py/geometry/GDMLtoCDB.py 2013-06-25 09:57:43 +0000 @@ -68,8 +68,9 @@ generated automatically """ self.config = Configreader() - self.wsdlurl = None - self.geometry_cdb = cdb.GeometrySuperMouse() + self.wsdlurl = self.config.cdb_upload_url+\ + self.config.geometry_upload_wsdl + self.geometry_cdb = cdb.GeometrySuperMouse(self.wsdlurl) self.textfile = textfile self.text = "" if type(notes) != str:
working from onrec01. From MAUS command line we are running
python upload_geometry.py --configuration_file Config.py
where Config.py has following contents
simulation_geometry_filename = "/data/neutrino02/rbayes/MICE/devel_maus/maus/files/geometry/download/ParentGeometryFile.dat" accumulate_tracks = 1 geant4_visualisation = True verbose_level = 4 spill_generator_number_of_spills = 10 # simulation_geometry_filename = "files/geometry/download/RotatedGeometryFile.dat" # configuration database cdb_upload_url = "http://preprodcdb.mice.rl.ac.uk" # target URL for configuration database uploads TestServer::http://rgma19.pp.rl.ac.uk:8080/cdb/ # cdb_download_url = "http://rgma19.pp.rl.ac.uk:8080/cdb/" # target URL for configuration database downloads # geometry download geometry_download_wsdl = "geometry?wsdl" # name of the web service used for downloads geometry_download_directory = "%s/files/geometry/download" % os.environ.get("MAUS_ROOT_DIR") # name of the local directory where downloads will be placed geometry_download_by = 'current' # choose 'run_number' to download by run number, 'current' to use # the currently valid geometry or 'id' to use the cdb internal id # (e.g. if it is desired to access an old version of a particular # geometry) geometry_download_run_number = 0 geometry_download_id = 0 geometry_download_cleanup = True # set to True to clean up after download g4_step_max = 5.0 # this is the value which shall be placed in the Mice Modules which have been translated from CAD # geometry upload geometry_upload_wsdl = "" # "/cdb/geometrySuperMouse.wsdl" # name of the web service used for uploads geometry_upload_directory = "%s/files/geometry/upload" % os.environ.get("MAUS_ROOT_DIR") # name of the local directory where uploads are drawn from geometry_upload_note = "New step IV geometry includes tracker but not placement of tracker. Does include placement of TOFs, KL, Ckov, and EMR. Now uses the correct materials for beamline elements." # note, pushed to the server to describe the geometry. A note must be specified here (default will throw an exception). geometry_upload_valid_from = "2013-06-08 18:00:00" # date-time in format like: that the specified installation was made in the experiment. A date-time must be specified here (default will throw an exception). technical_drawing_name = "TD-1189-3004" #name and version of the technical drawing from which the CAD model came from. geometry_upload_cleanup = False # set to True to clean up after upload #dates need to get geomtry ids get_ids_start_time = "2012-01-01 09:00:00" get_ids_stop_time = "2013-10-11 09:00:00" get_ids_create_file = False #get beamline information # This executable will give the run numbers of the runs which the CDB has information on. # The information is the magnet currents, reasons for run and other information which # is specific to that run. When downloading a geometry by run number the beamline # information is merged with the geometrical infomation. Options for querying # beamline information are; 'all_entries' returns a list of all run numbers with beamline info. # 'run_number' prints whether info is held for this run number or not. # 'dates' returns a list of run numbers with info during specified time period. get_beamline_by = "dates" get_beamline_run_number = "" get_beamline_start_time = "2009-10-15 09:00:00" get_beamline_stop_time = "2013-10-11 09:00:00"
Updated by Rogers, Chris over 9 years ago
Problem was that
- the constructor takes a url which overloads the environment variable/default and is the url of the server
- set_url(...) takes a url which is the url of the wsdl
We were calling both the constructor and set_url, with the same url. So always one or the other url would throw an exception, resulting in failure.
Propose set_url(...) should be deprecated and constructor should be the way to do it.
Updated by Rogers, Chris over 9 years ago
Still having trouble - I do
python upload_geometry.py --configuration_file Config.py "http://configdba.micenet.rl.ac.uk" Traceback (most recent call last): File "upload_geometry.py", line 72, in <module> main() File "upload_geometry.py", line 59, in main (ul_dir, str(configuration.geometry_upload_note)) File "/home/mice/MAUS/.maus_release/src/common_py/geometry/GDMLtoCDB.py", line 74, in __init__ self.geometry_cdb = cdb.GeometrySuperMouse(self.wsdlurl) File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry_supermouse.py", line 40, in __init__ "/cdb/geometrySuperMouse?wsdl") File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry.py", line 44, in __init__ super(Geometry, self).__init__(url, wsdl_dir) File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_base.py", line 39, in __init__ _cdb_wsdl) cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at http://configdba.micenet.rl.ac.uk/cdb/geometrySuperMouse?wsdl
nmap responds with
nmap configdba -P0 -p80 Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2013-06-27 11:17 BST Interesting ports on configdba.micenet.rl.ac.uk (172.16.246.25): PORT STATE SERVICE 80/tcp filtered http Nmap finished: 1 IP address (1 host up) scanned in 0.006 seconds
What am I doing wrong now?
Updated by Wilson, Antony over 9 years ago
filtered = behind firewall
configdba.micenet.rl.ac.uk is listening on port 8080. I've just updated iptables so you can now talk to it on port 80 as well
Updated by Rogers, Chris over 9 years ago
You're right, I didn't spot it - but I didn't see anywhere in config db area or online area a requirement to talk on this port or that port... (or even what URL to use)
Updated by Rogers, Chris over 9 years ago
ps: upload completed now
Ryan, I made a new install of MAUS on onrec01 /home/mice/MAUS/.maus_cdb
for doing cdb uploads. Can you please use this and not the production install for doing cdb uploads.
Updated by Rogers, Chris over 9 years ago
- Status changed from Open to Closed
- % Done changed from 0 to 100
Closed the issue, Antony may want to raise something under CDB...