Project

General

Profile

Support #1299

Failed to upload geometry to configuration database

Added by Rogers, Chris almost 11 years ago. Updated almost 11 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Wilson, Antony
Category:
Geometry
Target version:
Start date:
25 June 2013
Due date:
% Done:

100%

Estimated time:
Workflow:
New Issue

Description

We are trying to upload a geometry to the configuration database. As a starting point we decided to upload to either rgma (old test CDB server) or preprod, as it is clear we don't know what we are doing. I should say we tried several permutations on the URI without success. Email trail below:

Hi Chris,

So you know and before I forget, this was my last communication with Antony prior to leaving.

Also, I have the latest version of the geometry uploaded to ".maus_release/files/geometry/upload". Executing the "upload_geometry" command that we were running will post the files in that directory to the CDB. Probably an unnecessary point but I thought I should tell you that the files are there and that they are legitimate.

Cheers,
Ryan

Sent from my iPhone

Begin forwarded message:

From: "antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>" <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>>
Date: 25 June, 2013 11:34:39 AM GMT+01:00
To: Ryan Bayes <Ryan.Bayes@glasgow.ac.uk<mailto:Ryan.Bayes@glasgow.ac.uk>>
Cc: "Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>" <Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>>
Subject: RE: issue with geometry upload from miceonrec01

you can use http://preprodcdb.mice.rl.ac.uk:8080
NB at some point this will switch to port 80

This is a new machine and I'm not sure if it is visible externally yet, the plan is to open port 80 to the outside world

Regards
Antony
________________________________
From: Ryan Bayes [Ryan.Bayes@glasgow.ac.uk<mailto:Ryan.Bayes@glasgow.ac.uk>]
Sent: 25 June 2013 11:25
To: Wilson, Antony (STFC,RAL,SC)
Cc: Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>
Subject: Re: issue with geometry upload from miceonrec01

Thank you. That is helpful.

Another question: Mat suggested using the test server "rgma19.pp.rl.ac.uk<http://pp.rl.ac.uk>". Do you know if this server is indeed in operation.

Thanks again,
Ryan

On 2013-06-25, at 10:29 AM, <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk><mailto:antony.wilson@stfc.ac.uk>> wrote:

cdb.mice.ac.uk<http://cdb.mice.ac.uk><http://cdb.mice.ac.uk><http://cdb.mice.ac.uk> is the default server, not the default upload server, by default you get read only access. Matt has a machine in the control room that he uses for upload. It should be configured to use configdb.micenet.rl.ac.uk

Regards
Antony

> From: Ryan Bayes [Ryan.Bayes@glasgow.ac.uk]
> Sent: 25 June 2013 10:19
> To: Wilson, Antony (STFC,RAL,SC)
> Cc: Matthew.Littlefield@brunel.ac.uk
> Subject: Re: issue with geometry upload from miceonrec01
> 
> But I am now confused.
> 
> I was trying from the mice control room. I was under the impression that would put me behind the firewall.
> 
> Also, cdb.mice.ac.uk<http://cdb.mice.ac.uk> is the default upload server in the CDB software as distributed. I learned from Mat that there is a way to get around this but the correct server name is required, and I am not sure what that is.
> 
> Respectfully,
> Ryan
> 
> 
> 
> On 2013-06-24, at 8:21 AM, <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>> <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>> wrote:
> 
> Hi Ryan
> 
> cdb.mice.rl.ac.uk provides read only access to the cdb. The cdb super mouse web service is running on a machine in the control room and has restricted access. If you need to upload a geometry to the production system then Mat should be able to help
> 
> Regards
> Antony
> 
> 
> 
> -----Original Message-----
> From: Ryan Bayes [mailto:Ryan.Bayes@glasgow.ac.uk]
> Sent: 23 June 2013 17:55
> To: Wilson, Antony (STFC,RAL,SC); Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>
> Subject: issue with geometry upload from miceonrec01
> 
> Hello Antony,
> 
> I know that you have moved on to bigger and brighter things, but I hoped you could help me with a problem or direct me to somebody else who could.
> 
> I have been trying to upload a geometry to the MICE configuration data base, and I get the following feed back
> 
> 
> [miceonrec01] utilities > ./upload_geometry.py --configuration_file=Config.py Traceback (most recent call last):
>   File "./upload_geometry.py", line 72, in <module>
>     main()
>   File "./upload_geometry.py", line 59, in main
>     (ul_dir, str(configuration.geometry_upload_note))
>   File "/home/mice/MAUS/.maus_release/src/common_py/geometry/GDMLtoCDB.py", line 72, in __init__
>     self.geometry_cdb = cdb.GeometrySuperMouse()
>   File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry_supermouse.py", line 40, in __init__
>     "/cdb/geometrySuperMouse?wsdl")
>   File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry.py", line 44, in __init__
>     super(Geometry, self).__init__(url, wsdl_dir)
>   File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_base.py", line 39, in __init__
>     _cdb_wsdl)
> cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at http://cdb.mice.rl.ac.uk/cdb/geometrySuperMouse?wsdl
> 
> 
> The contents of the configuration file do not matter for the purpose of this problem since it seems to be in the ability of the program
> to contact the server (which is defined in the SuperMouse method of CDB). Is there a different machine or user I should be doing this as?
> Is there a different problem I should be aware of?
> 
> Thank you,
> Ryan Bayes
> --
> Scanned by iCritical.
> 
> 
#1

Updated by Rogers, Chris almost 11 years ago

Few more comments. We are using MAUS 0.5.4, with one modification to GDMLtoCDB.py to stop cdb defaulting to URL from an environment variable; diff output:

=== modified file 'src/common_py/geometry/GDMLtoCDB.py'
--- src/common_py/geometry/GDMLtoCDB.py    2013-02-13 14:49:22 +0000
+++ src/common_py/geometry/GDMLtoCDB.py    2013-06-25 09:57:43 +0000
@@ -68,8 +68,9 @@
                         generated automatically
         """ 
         self.config = Configreader()
-        self.wsdlurl = None
-        self.geometry_cdb = cdb.GeometrySuperMouse()
+        self.wsdlurl = self.config.cdb_upload_url+\
+                                                self.config.geometry_upload_wsdl
+        self.geometry_cdb = cdb.GeometrySuperMouse(self.wsdlurl)
         self.textfile = textfile
         self.text = "" 
         if type(notes) != str:

working from onrec01. From MAUS command line we are running

python upload_geometry.py --configuration_file Config.py

where Config.py has following contents

simulation_geometry_filename = "/data/neutrino02/rbayes/MICE/devel_maus/maus/files/geometry/download/ParentGeometryFile.dat" 
accumulate_tracks = 1
geant4_visualisation = True
verbose_level = 4
spill_generator_number_of_spills = 10

# simulation_geometry_filename = "files/geometry/download/RotatedGeometryFile.dat" 

# configuration database
cdb_upload_url = "http://preprodcdb.mice.rl.ac.uk" 
# target URL for configuration database uploads TestServer::http://rgma19.pp.rl.ac.uk:8080/cdb/
# cdb_download_url = "http://rgma19.pp.rl.ac.uk:8080/cdb/" # target URL for configuration database downloads

# geometry download
geometry_download_wsdl = "geometry?wsdl" # name of the web service used for downloads
geometry_download_directory   = "%s/files/geometry/download" % os.environ.get("MAUS_ROOT_DIR") # name of the local directory where downloads will be placed
geometry_download_by = 'current' # choose 'run_number' to download by run number, 'current' to use
                                    # the currently valid geometry or 'id' to use the cdb internal id 
                                    # (e.g. if it is desired to access an old version of a particular
                                    # geometry)
geometry_download_run_number = 0
geometry_download_id = 0
geometry_download_cleanup = True # set to True to clean up after download
g4_step_max = 5.0 # this is the value which shall be placed in the Mice Modules which have been translated from CAD

# geometry upload
geometry_upload_wsdl = "" # "/cdb/geometrySuperMouse.wsdl" # name of the web service used for uploads
geometry_upload_directory = "%s/files/geometry/upload" % os.environ.get("MAUS_ROOT_DIR") # name of the local directory where uploads are drawn from
geometry_upload_note = "New step IV geometry includes tracker but not placement of tracker. Does include placement of TOFs, KL, Ckov, and EMR. Now uses the correct materials for beamline elements." 
# note, pushed to the server to describe the geometry. A note must be specified here (default will throw an exception).
geometry_upload_valid_from = "2013-06-08 18:00:00" # date-time in format like: that the specified installation was made in the experiment. A date-time must be specified here (default will throw an exception).
technical_drawing_name = "TD-1189-3004" #name and version of the technical drawing from which the CAD model came from.
geometry_upload_cleanup = False # set to True to clean up after upload

#dates need to get geomtry ids
get_ids_start_time = "2012-01-01 09:00:00" 
get_ids_stop_time = "2013-10-11 09:00:00" 
get_ids_create_file = False

#get beamline information
# This executable will give the run numbers of the runs which the CDB has information on.
# The information is the magnet currents, reasons for run and other information which
# is specific to that run. When downloading a geometry by run number the beamline 
# information is merged with the geometrical infomation. Options for querying
# beamline information are; 'all_entries' returns a list of all run numbers with beamline info.
#                           'run_number'  prints whether info is held for this run number or not.
#                           'dates'       returns a list of run numbers with info during specified time period.
get_beamline_by = "dates" 
get_beamline_run_number = "" 
get_beamline_start_time = "2009-10-15 09:00:00" 
get_beamline_stop_time = "2013-10-11 09:00:00" 
#2

Updated by Rogers, Chris almost 11 years ago

Problem was that

  • the constructor takes a url which overloads the environment variable/default and is the url of the server
  • set_url(...) takes a url which is the url of the wsdl

We were calling both the constructor and set_url, with the same url. So always one or the other url would throw an exception, resulting in failure.

Propose set_url(...) should be deprecated and constructor should be the way to do it.

#3

Updated by Rogers, Chris almost 11 years ago

Still having trouble - I do

python upload_geometry.py --configuration_file Config.py
"http://configdba.micenet.rl.ac.uk" 
Traceback (most recent call last):
  File "upload_geometry.py", line 72, in <module>
    main()
  File "upload_geometry.py", line 59, in main
    (ul_dir, str(configuration.geometry_upload_note))
  File "/home/mice/MAUS/.maus_release/src/common_py/geometry/GDMLtoCDB.py", line 74, in __init__
    self.geometry_cdb = cdb.GeometrySuperMouse(self.wsdlurl)
  File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry_supermouse.py", line 40, in __init__
    "/cdb/geometrySuperMouse?wsdl")
  File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry.py", line 44, in __init__
    super(Geometry, self).__init__(url, wsdl_dir)
  File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_base.py", line 39, in __init__
    _cdb_wsdl)
cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at http://configdba.micenet.rl.ac.uk/cdb/geometrySuperMouse?wsdl

nmap responds with

nmap configdba -P0 -p80

Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2013-06-27 11:17 BST
Interesting ports on configdba.micenet.rl.ac.uk (172.16.246.25):
PORT   STATE    SERVICE
80/tcp filtered http

Nmap finished: 1 IP address (1 host up) scanned in 0.006 seconds

What am I doing wrong now?

#4

Updated by Wilson, Antony almost 11 years ago

filtered = behind firewall

configdba.micenet.rl.ac.uk is listening on port 8080. I've just updated iptables so you can now talk to it on port 80 as well

#5

Updated by Rogers, Chris almost 11 years ago

You're right, I didn't spot it - but I didn't see anywhere in config db area or online area a requirement to talk on this port or that port... (or even what URL to use)

#6

Updated by Rogers, Chris almost 11 years ago

ps: upload completed now

Ryan, I made a new install of MAUS on onrec01 /home/mice/MAUS/.maus_cdb for doing cdb uploads. Can you please use this and not the production install for doing cdb uploads.

#7

Updated by Rogers, Chris almost 11 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Closed the issue, Antony may want to raise something under CDB...

Also available in: Atom PDF