Project

General

Profile

Support #1299

Failed to upload geometry to configuration database

Added by Rogers, Chris over 9 years ago. Updated over 9 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Wilson, Antony
Category:
Geometry
Target version:
Start date:
25 June 2013
Due date:
% Done:

100%

Estimated time:
Workflow:
New Issue

Description

We are trying to upload a geometry to the configuration database. As a starting point we decided to upload to either rgma (old test CDB server) or preprod, as it is clear we don't know what we are doing. I should say we tried several permutations on the URI without success. Email trail below:

Hi Chris,

So you know and before I forget, this was my last communication with Antony prior to leaving.

Also, I have the latest version of the geometry uploaded to ".maus_release/files/geometry/upload". Executing the "upload_geometry" command that we were running will post the files in that directory to the CDB. Probably an unnecessary point but I thought I should tell you that the files are there and that they are legitimate.

Cheers,
Ryan

Sent from my iPhone

Begin forwarded message:

From: "antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>" <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>>
Date: 25 June, 2013 11:34:39 AM GMT+01:00
To: Ryan Bayes <Ryan.Bayes@glasgow.ac.uk<mailto:Ryan.Bayes@glasgow.ac.uk>>
Cc: "Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>" <Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>>
Subject: RE: issue with geometry upload from miceonrec01

you can use http://preprodcdb.mice.rl.ac.uk:8080
NB at some point this will switch to port 80

This is a new machine and I'm not sure if it is visible externally yet, the plan is to open port 80 to the outside world

Regards
Antony
________________________________
From: Ryan Bayes [Ryan.Bayes@glasgow.ac.uk<mailto:Ryan.Bayes@glasgow.ac.uk>]
Sent: 25 June 2013 11:25
To: Wilson, Antony (STFC,RAL,SC)
Cc: Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>
Subject: Re: issue with geometry upload from miceonrec01

Thank you. That is helpful.

Another question: Mat suggested using the test server "rgma19.pp.rl.ac.uk<http://pp.rl.ac.uk>". Do you know if this server is indeed in operation.

Thanks again,
Ryan

On 2013-06-25, at 10:29 AM, <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk><mailto:antony.wilson@stfc.ac.uk>> wrote:

cdb.mice.ac.uk<http://cdb.mice.ac.uk><http://cdb.mice.ac.uk><http://cdb.mice.ac.uk> is the default server, not the default upload server, by default you get read only access. Matt has a machine in the control room that he uses for upload. It should be configured to use configdb.micenet.rl.ac.uk

Regards
Antony

> From: Ryan Bayes [Ryan.Bayes@glasgow.ac.uk]
> Sent: 25 June 2013 10:19
> To: Wilson, Antony (STFC,RAL,SC)
> Cc: Matthew.Littlefield@brunel.ac.uk
> Subject: Re: issue with geometry upload from miceonrec01
> 
> But I am now confused.
> 
> I was trying from the mice control room. I was under the impression that would put me behind the firewall.
> 
> Also, cdb.mice.ac.uk<http://cdb.mice.ac.uk> is the default upload server in the CDB software as distributed. I learned from Mat that there is a way to get around this but the correct server name is required, and I am not sure what that is.
> 
> Respectfully,
> Ryan
> 
> 
> 
> On 2013-06-24, at 8:21 AM, <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>> <antony.wilson@stfc.ac.uk<mailto:antony.wilson@stfc.ac.uk>> wrote:
> 
> Hi Ryan
> 
> cdb.mice.rl.ac.uk provides read only access to the cdb. The cdb super mouse web service is running on a machine in the control room and has restricted access. If you need to upload a geometry to the production system then Mat should be able to help
> 
> Regards
> Antony
> 
> 
> 
> -----Original Message-----
> From: Ryan Bayes [mailto:Ryan.Bayes@glasgow.ac.uk]
> Sent: 23 June 2013 17:55
> To: Wilson, Antony (STFC,RAL,SC); Matthew.Littlefield@brunel.ac.uk<mailto:Matthew.Littlefield@brunel.ac.uk>
> Subject: issue with geometry upload from miceonrec01
> 
> Hello Antony,
> 
> I know that you have moved on to bigger and brighter things, but I hoped you could help me with a problem or direct me to somebody else who could.
> 
> I have been trying to upload a geometry to the MICE configuration data base, and I get the following feed back
> 
> 
> [miceonrec01] utilities > ./upload_geometry.py --configuration_file=Config.py Traceback (most recent call last):
>   File "./upload_geometry.py", line 72, in <module>
>     main()
>   File "./upload_geometry.py", line 59, in main
>     (ul_dir, str(configuration.geometry_upload_note))
>   File "/home/mice/MAUS/.maus_release/src/common_py/geometry/GDMLtoCDB.py", line 72, in __init__
>     self.geometry_cdb = cdb.GeometrySuperMouse()
>   File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry_supermouse.py", line 40, in __init__
>     "/cdb/geometrySuperMouse?wsdl")
>   File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry.py", line 44, in __init__
>     super(Geometry, self).__init__(url, wsdl_dir)
>   File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_base.py", line 39, in __init__
>     _cdb_wsdl)
> cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at http://cdb.mice.rl.ac.uk/cdb/geometrySuperMouse?wsdl
> 
> 
> The contents of the configuration file do not matter for the purpose of this problem since it seems to be in the ability of the program
> to contact the server (which is defined in the SuperMouse method of CDB). Is there a different machine or user I should be doing this as?
> Is there a different problem I should be aware of?
> 
> Thank you,
> Ryan Bayes
> --
> Scanned by iCritical.
> 
> 
#1

Updated by Rogers, Chris over 9 years ago

Few more comments. We are using MAUS 0.5.4, with one modification to GDMLtoCDB.py to stop cdb defaulting to URL from an environment variable; diff output:

=== modified file 'src/common_py/geometry/GDMLtoCDB.py'
--- src/common_py/geometry/GDMLtoCDB.py    2013-02-13 14:49:22 +0000
+++ src/common_py/geometry/GDMLtoCDB.py    2013-06-25 09:57:43 +0000
@@ -68,8 +68,9 @@
                         generated automatically
         """ 
         self.config = Configreader()
-        self.wsdlurl = None
-        self.geometry_cdb = cdb.GeometrySuperMouse()
+        self.wsdlurl = self.config.cdb_upload_url+\
+                                                self.config.geometry_upload_wsdl
+        self.geometry_cdb = cdb.GeometrySuperMouse(self.wsdlurl)
         self.textfile = textfile
         self.text = "" 
         if type(notes) != str:

working from onrec01. From MAUS command line we are running

python upload_geometry.py --configuration_file Config.py

where Config.py has following contents

simulation_geometry_filename = "/data/neutrino02/rbayes/MICE/devel_maus/maus/files/geometry/download/ParentGeometryFile.dat" 
accumulate_tracks = 1
geant4_visualisation = True
verbose_level = 4
spill_generator_number_of_spills = 10

# simulation_geometry_filename = "files/geometry/download/RotatedGeometryFile.dat" 

# configuration database
cdb_upload_url = "http://preprodcdb.mice.rl.ac.uk" 
# target URL for configuration database uploads TestServer::http://rgma19.pp.rl.ac.uk:8080/cdb/
# cdb_download_url = "http://rgma19.pp.rl.ac.uk:8080/cdb/" # target URL for configuration database downloads

# geometry download
geometry_download_wsdl = "geometry?wsdl" # name of the web service used for downloads
geometry_download_directory   = "%s/files/geometry/download" % os.environ.get("MAUS_ROOT_DIR") # name of the local directory where downloads will be placed
geometry_download_by = 'current' # choose 'run_number' to download by run number, 'current' to use
                                    # the currently valid geometry or 'id' to use the cdb internal id 
                                    # (e.g. if it is desired to access an old version of a particular
                                    # geometry)
geometry_download_run_number = 0
geometry_download_id = 0
geometry_download_cleanup = True # set to True to clean up after download
g4_step_max = 5.0 # this is the value which shall be placed in the Mice Modules which have been translated from CAD

# geometry upload
geometry_upload_wsdl = "" # "/cdb/geometrySuperMouse.wsdl" # name of the web service used for uploads
geometry_upload_directory = "%s/files/geometry/upload" % os.environ.get("MAUS_ROOT_DIR") # name of the local directory where uploads are drawn from
geometry_upload_note = "New step IV geometry includes tracker but not placement of tracker. Does include placement of TOFs, KL, Ckov, and EMR. Now uses the correct materials for beamline elements." 
# note, pushed to the server to describe the geometry. A note must be specified here (default will throw an exception).
geometry_upload_valid_from = "2013-06-08 18:00:00" # date-time in format like: that the specified installation was made in the experiment. A date-time must be specified here (default will throw an exception).
technical_drawing_name = "TD-1189-3004" #name and version of the technical drawing from which the CAD model came from.
geometry_upload_cleanup = False # set to True to clean up after upload

#dates need to get geomtry ids
get_ids_start_time = "2012-01-01 09:00:00" 
get_ids_stop_time = "2013-10-11 09:00:00" 
get_ids_create_file = False

#get beamline information
# This executable will give the run numbers of the runs which the CDB has information on.
# The information is the magnet currents, reasons for run and other information which
# is specific to that run. When downloading a geometry by run number the beamline 
# information is merged with the geometrical infomation. Options for querying
# beamline information are; 'all_entries' returns a list of all run numbers with beamline info.
#                           'run_number'  prints whether info is held for this run number or not.
#                           'dates'       returns a list of run numbers with info during specified time period.
get_beamline_by = "dates" 
get_beamline_run_number = "" 
get_beamline_start_time = "2009-10-15 09:00:00" 
get_beamline_stop_time = "2013-10-11 09:00:00" 
#2

Updated by Rogers, Chris over 9 years ago

Problem was that

  • the constructor takes a url which overloads the environment variable/default and is the url of the server
  • set_url(...) takes a url which is the url of the wsdl

We were calling both the constructor and set_url, with the same url. So always one or the other url would throw an exception, resulting in failure.

Propose set_url(...) should be deprecated and constructor should be the way to do it.

#3

Updated by Rogers, Chris over 9 years ago

Still having trouble - I do

python upload_geometry.py --configuration_file Config.py
"http://configdba.micenet.rl.ac.uk" 
Traceback (most recent call last):
  File "upload_geometry.py", line 72, in <module>
    main()
  File "upload_geometry.py", line 59, in main
    (ul_dir, str(configuration.geometry_upload_note))
  File "/home/mice/MAUS/.maus_release/src/common_py/geometry/GDMLtoCDB.py", line 74, in __init__
    self.geometry_cdb = cdb.GeometrySuperMouse(self.wsdlurl)
  File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry_supermouse.py", line 40, in __init__
    "/cdb/geometrySuperMouse?wsdl")
  File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_geometry.py", line 44, in __init__
    super(Geometry, self).__init__(url, wsdl_dir)
  File "/home/mice/MAUS/.maus_release/third_party/install/lib/python2.7/site-packages/cdb/_base.py", line 39, in __init__
    _cdb_wsdl)
cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at http://configdba.micenet.rl.ac.uk/cdb/geometrySuperMouse?wsdl

nmap responds with

nmap configdba -P0 -p80

Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2013-06-27 11:17 BST
Interesting ports on configdba.micenet.rl.ac.uk (172.16.246.25):
PORT   STATE    SERVICE
80/tcp filtered http

Nmap finished: 1 IP address (1 host up) scanned in 0.006 seconds

What am I doing wrong now?

#4

Updated by Wilson, Antony over 9 years ago

filtered = behind firewall

configdba.micenet.rl.ac.uk is listening on port 8080. I've just updated iptables so you can now talk to it on port 80 as well

#5

Updated by Rogers, Chris over 9 years ago

You're right, I didn't spot it - but I didn't see anywhere in config db area or online area a requirement to talk on this port or that port... (or even what URL to use)

#6

Updated by Rogers, Chris over 9 years ago

ps: upload completed now

Ryan, I made a new install of MAUS on onrec01 /home/mice/MAUS/.maus_cdb for doing cdb uploads. Can you please use this and not the production install for doing cdb uploads.

#7

Updated by Rogers, Chris over 9 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Closed the issue, Antony may want to raise something under CDB...

Also available in: Atom PDF