Project

General

Profile

Bug #1306

preprodcb non communicative

Added by Rogers, Chris over 10 years ago. Updated over 10 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Rogers, Chris
Category:
Server
Start date:
10 July 2013
Due date:
% Done:

100%

Estimated time:

Description

See notes at:

http://micewww.pp.rl.ac.uk/projects/computing-software/wiki/PC_20130710

and:

Doing a little investigation - it does not appear to be a firewall
issue. I can contact preprodcdb on port 80 both inside and outside the
firewall (using a thing Antony showed me a couple of weeks ago).

When working outside the firewall, I use data connection on my mobile
phone as an internet access point (to prove I am really outside the
firewall).

> $ nmap preprodcdb.mice.rl.ac.uk -p80
Starting Nmap 5.21 ( http://nmap.org ) at 2013-07-09 09:07 BST
Nmap scan report for preprodcdb.mice.rl.ac.uk (130.246.92.153)
Host is up (0.0023s latency).
rDNS record for 130.246.92.153: heplnv153.pp.rl.ac.uk
PORT   STATE SERVICE
80/tcp open  http

Nmap done: 1 IP address (1 host up) scanned in 0.04 seconds

However, when I try to contact preprod using the cdb python library I
get an error, regardless of whether I am inside the firewall or not

> $ python
>>> from cdb import Beamline
>>> bl = Beamline('http://preprodcdb.mice.rl.ac.uk')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File
"/home/cr67/MAUS/maus_merge/third_party/install/lib/python2.7/site-packages/cdb/_beamline.py",
line 45, in __init__
    super(Beamline, self).__init__(url, wsdl_dir)
  File
"/home/cr67/MAUS/maus_merge/third_party/install/lib/python2.7/site-packages/cdb/_base.py",
line 39, in __init__
    _cdb_wsdl)
cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at
http://preprodcdb.mice.rl.ac.uk/cdb/beamline?wsdl

I believe this indicates a problem with the wsdl service on the
server, not the firewall. Not sure if that helps, more locates who to
blame.

#1

Updated by Rogers, Chris over 10 years ago

cdb.log indicates last access was at

'2013-06-28 17:27:18.197', 'SS2', 'COLD_READY');
2013-06-28 17:27:18,332 [http-8080-Processor25] DEBUG mice.cdb.service.DBAccess - Connection closed
#2

Updated by Rogers, Chris over 10 years ago

Going through the docs at

http://micewww.pp.rl.ac.uk/projects/configdb/wiki/PublicInterface

--

  • yum install xml-commons-apis.x86_64 tomcat5 postgresql-jdbc

Nothing done

  • ln -s /usr/share/java/postgresql-jdbc.jar /var/lib/tomcat5/common/lib/postgresql-jdbc.jar

link exists

  • ln -s /usr/share/java/xalan-j2.jar /var/lib/tomcat5/common/lib/xalan-j2.jar

link exists

  • emacs /etc/tomcat5/server.xml; port 8080 -> 4443

ERROR there was no reference to 4443 in the list

  • copy the war file to /var/lib/tomcat5/webapps/

There is a file cdb.war at this location

  • copy the log4j.properties file to /opt/mice/etc/cdb-server/

There is log4j file at this location; note it has been edited

  • configure Configuration DB

/opt/mice/etc/cdb-server/cdb.props

ERROR db.url=jdbc:postgresql://localhost/
ERROR ls -lah cdb.props yields
-rw-r--r-- 1 root root 160 Feb 6 13:39 cdb.props

  • create the Configuration DB log file directory

directory exists with correct permissions

  • customise logging levels - log level line is commented, I could uncomment
  • get Tomcat up and running as a service

/sbin/service tomcat5 status
/etc/init.d/tomcat5 is already running (2812)

  • configure Apache to forward messages to Tomcat

I don't know what this means

Attempted Fix

I copied
/etc/tomcat5/server.xml

to

/etc/tomcat5/server.xml.bak

I edited the file

/etc/tomcat5/server.xml

to change the port for http from 8080 to 4443

I restarted tomcat

/sbin/service/tomcat5 restart

Now I get from

http://preprodcdb.mice.rl.ac.uk/cdb/beamline?wsdl

ERROR

The requested URL could not be retrieved

While trying to retrieve the URL: http://preprodcdb.mice.rl.ac.uk/cdb/beamline?wsdl

The following error was encountered:

    Connection to 130.246.92.153 Failed 

The system returned:

    (111) Connection refused

The remote host or network may be down. Please try the request again.

Your cache administrator is netsdl@stfc.ac.uk.
Generated on Thu, 11 Jul 2013 16:55:17 +0100

I restart tomcat

/sbin/service/tomcat5 restart

and I get back to the other error message

HTTP Status 404 - /cdb/beamline

type Status report

message /cdb/beamline

description The requested resource (/cdb/beamline) is not available.
Apache Tomcat/5.5.23

So I believe tomcat is available and listening on port 8080

#3

Updated by Rogers, Chris over 10 years ago

I also tried adding a port to

/opt/mice/etc/cdb-server/cdb.props

both

db.url=jdbc:postgresql://localhost:8080/
db.url=jdbc:postgresql://localhost:5432/

each time followed by tomcat restart; neither thing helped.

#4

Updated by Rogers, Chris over 10 years ago

I have to go home. I am on the server running commands like from:

http://tomcat.apache.org/tomcat-6.0-doc/manager-howto.html#Configuring_Manager_Application_Access

trying to figure out how to list tomcat web applications to investigate the layer between tomcat and postgres.

#5

Updated by Rogers, Chris over 10 years ago

Also from various emails running in parallel

The error is

HTTP Status 404 - /cdb/beamline.wsdl

type Status report

message /cdb/beamline.wsdl

description The requested resource (/cdb/beamline.wsdl) is not available.
Apache Tomcat/5.5.23

where the last line indicates to me that this error message is produced by Tomcat; so server knows something should be presented there but somehow there is a problem in the backend.

On server side I see a log file /var/log/tomcat5/catalina.out indicating something really wrong in the wsdl, below. I believe this means the SOAP layer is failing. If I try a wget from onboard the server (http_proxy="" wget http://127.0.0.1/cdb/beamline.wsdl) I get a 404. This seems to be what it is tripping on.

10-Jul-2013 16:05:31 org.apache.catalina.startup.Catalina start
INFO: Server startup in 2079 ms
mice.cdb.client.CDBException: Unable to create Beamline. Failed to access the WSDL at: http://127.0.0.1:8080/cdb/beamline?wsdl. It failed with:
        http://127.0.0.1:8080/cdb/beamline?wsdl.
        at mice.cdb.client.beamline.Beamline.init(Beamline.java:90)
        at mice.cdb.client.beamline.Beamline.<init>(Beamline.java:68)
        at mice.cdb.gwt.server.SearchServiceImpl.<init>(SearchServiceImpl.java:35)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
        at java.lang.Class.newInstance0(Class.java:374)
        at java.lang.Class.newInstance(Class.java:327)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1089)
        at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:791)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:127)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:542)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
        at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
        at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
        at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
        at java.lang.Thread.run(Thread.java:679)
Caused by: javax.xml.ws.WebServiceException: Failed to access the WSDL at: http://127.0.0.1:8080/cdb/beamline?wsdl. It failed with:
        http://127.0.0.1:8080/cdb/beamline?wsdl.
        at com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.tryWithMex(RuntimeWSDLParser.java:151)
        at com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:133)
        at com.sun.xml.internal.ws.client.WSServiceDelegate.parseWSDL(WSServiceDelegate.java:254)
        at com.sun.xml.internal.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:217)
        at com.sun.xml.internal.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:165)
        at com.sun.xml.internal.ws.spi.ProviderImpl.createServiceDelegate(ProviderImpl.java:93)
        at javax.xml.ws.Service.<init>(Service.java:76)
        at mice.cdb.service.beamline.BeamlineImplService.<init>(BeamlineImplService.java:42)
        at mice.cdb.client.beamline.Beamline.init(Beamline.java:87)
        ... 23 more
Caused by: java.io.FileNotFoundException: http://127.0.0.1:8080/cdb/beamline?wsdl
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1401)
        at java.net.URL.openStream(URL.java:1031)
        at com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.createReader(RuntimeWSDLParser.java:793)
        at com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.resolveWSDL(RuntimeWSDLParser.java:251)
        at com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:118)
#6

Updated by Rogers, Chris over 10 years ago

Editing the tomcat configuration stored at /usr/share/tomcat5/conf; I tried adding deployOnStartup to server.xml Host element; restarted tomcat but still was unable to contact Beamline.wsdl. I reverted the change to server.xml

#7

Updated by Rogers, Chris over 10 years ago

I tried installing the manager, using an xml at /usr/share/tomcat5/conf/Catalina/localhost/manager.xml like

<Context privileged="true" antiResourceLocking="false" 
         docBase="/usr/share/tomcat5/webapps/manager">
  <Valve className="org.apache.catalina.valves.RemoteAddrValve" 
         allow="127\.0\.0\.1" />
</Context>

But catalina.out reports

java.lang.IllegalArgumentException: Document base /usr/share/tomcat5/webapps/manager does not exist or is not a readable directory

indeed the directory does not exist. Documents indicate it comes by default so Antony must have removed it?

Edit to add: I notice that on my SL6 box manager is in a separate RPM to the rest of tomcat. Maybe the correct RPM is not installed?

#8

Updated by Rogers, Chris over 10 years ago

Pierrick sensibly wants some confirmation that the cdb is running in postgres. A little googling led me to

bash-3.2$ psql --username=postgres
psql (9.0.13)
Type "help" for help.

postgres=# \list
                                  List of databases
   Name    |   Owner    | Encoding | Collation  |   Ctype    |   Access privileges   
-----------+------------+----------+------------+------------+-----------------------
 cdb       | supermouse | UTF8     | en_GB.utf8 | en_GB.utf8 | 
 postgres  | postgres   | UTF8     | en_GB.utf8 | en_GB.utf8 | 
 template0 | postgres   | UTF8     | en_GB.utf8 | en_GB.utf8 | =c/postgres          +
           |            |          |            |            | postgres=CTc/postgres
 template1 | postgres   | UTF8     | en_GB.utf8 | en_GB.utf8 | postgres=CTc/postgres+
           |            |          |            |            | =c/postgres

I think this means postgres portion of cdb is running.

#9

Updated by Rogers, Chris over 10 years ago 

-bash-3.2$ sudo su postgres
[sudo] password for cr67: 
bash-3.2$ psql
psql (9.0.13)
Type "help" for help.

postgres=# \c cdb
You are now connected to database "cdb".
cdb=# \d
                      List of relations
 Schema |            Name            |   Type   |   Owner    
--------+----------------------------+----------+------------
 public | alarm                      | table    | supermouse
 public | alarmhistory               | table    | supermouse
 public | beamline                   | table    | supermouse
 public | beamlinetag                | table    | supermouse
 public | beamlinetag_id_seq         | sequence | supermouse
 public | beamlinetagged             | table    | supermouse
 public | cablingcontrol             | table    | supermouse
 public | cablingdetector            | table    | supermouse
 public | cablingdetector_id_seq     | sequence | supermouse
 public | cablingtracker             | table    | supermouse
 public | calibrationdetector        | table    | supermouse
 public | calibrationdetector_id_seq | sequence | supermouse
 public | calibrationtracker         | table    | supermouse
 public | channels                   | table    | supermouse
 public | controls                   | table    | supermouse
 public | coolingchannel             | table    | supermouse
 public | coolingchannel_id_seq      | sequence | supermouse
 public | coolingchannelcoil         | table    | supermouse
 public | coolingchannelcoiltagged   | table    | supermouse
 public | coolingchannelmagnet       | table    | supermouse
 public | coolingchannelmagnettagged | table    | supermouse
 public | coolingchanneltag          | table    | supermouse
 public | coolingchanneltag_id_seq   | sequence | supermouse
 public | currentstate               | table    | supermouse
 public | daq                        | table    | supermouse
 public | geometry                   | table    | supermouse
 public | geometry_id_seq            | sequence | supermouse
 public | isis                       | table    | supermouse
 public | isisbeam                   | table    | supermouse
 public | ldchosts                   | table    | supermouse
 public | magnet                     | table    | supermouse
 public | magnettagged               | table    | supermouse
 public | pvdata                     | table    | supermouse
 public | run                        | table    | supermouse
 public | runno                      | table    | postgres
 public | scalar                     | table    | supermouse
 public | statemachine               | table    | supermouse
 public | target                     | table    | supermouse
 public | targetblob                 | table    | supermouse
 public | targetfloat                | table    | supermouse
 public | targetinteger              | table    | supermouse
 public | targetold                  | table    | supermouse
 public | targetsetbcd               | table    | supermouse
 public | targetstring               | table    | supermouse
 public | targetuserdelay            | table    | supermouse
(45 rows)

cdb=# select * from alarm;

produces long list of tables

#10

Updated by Rogers, Chris over 10 years ago

Progress - I was able to get a read only copy running by wholesale copying the webapps directory from cdb.mice.rl.ac.uk across to preprodcdb. I noticed a difference

-bash-3.2$ ls /var/lib/tomcat5/webapps/cdb/
doc  index.html  META-INF  WEB-INF
-bash-3.2$ ls /var/lib/tomcat5/webapps/cdb/*-INF/
/var/lib/tomcat5/webapps/cdb/META-INF/:
MANIFEST.MF

/var/lib/tomcat5/webapps/cdb/WEB-INF/:
lib  LICENSE  sun-jaxws.xml  web.xml

whereas for the old webapps directory (which I moved to _webapps)

-bash-3.2$ ls /var/lib/tomcat5/_webapps/cdb/
doc  index.html

I can start digging into these *-INF directories in the morning...

#11

Updated by Rogers, Chris over 10 years ago

  • Status changed from Open to Closed
  • Assignee changed from Martyniak, Janusz to Rogers, Chris
  • % Done changed from 0 to 100

Let me just finish this off - I moved the cdb directory from the production cdb to the preprod cdb and edited the xml files to add in SuperMouse elements to the schema.

The old webapps is saved as /var/lib/tomcat5/webapps_preprod and a copy of production cdb is saved in /var/lib/tomcat5/webapps_preprod

It all seems to be working and no one has complained so I will close the issue.

Also available in: Atom PDF