Bug #1306

preprodcb non communicative

Added by Rogers, Chris about 9 years ago. Updated about 9 years ago.

Start date:
10 July 2013
Due date:
% Done:


Estimated time:


See notes at:


Doing a little investigation - it does not appear to be a firewall
issue. I can contact preprodcdb on port 80 both inside and outside the
firewall (using a thing Antony showed me a couple of weeks ago).

When working outside the firewall, I use data connection on my mobile
phone as an internet access point (to prove I am really outside the

> $ nmap -p80
Starting Nmap 5.21 ( ) at 2013-07-09 09:07 BST
Nmap scan report for (
Host is up (0.0023s latency).
rDNS record for
80/tcp open  http

Nmap done: 1 IP address (1 host up) scanned in 0.04 seconds

However, when I try to contact preprod using the cdb python library I
get an error, regardless of whether I am inside the firewall or not

> $ python
>>> from cdb import Beamline
>>> bl = Beamline('')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
line 45, in __init__
    super(Beamline, self).__init__(url, wsdl_dir)
line 39, in __init__
cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at

I believe this indicates a problem with the wsdl service on the
server, not the firewall. Not sure if that helps, more locates who to


Updated by Rogers, Chris about 9 years ago

cdb.log indicates last access was at

'2013-06-28 17:27:18.197', 'SS2', 'COLD_READY');
2013-06-28 17:27:18,332 [http-8080-Processor25] DEBUG mice.cdb.service.DBAccess - Connection closed

Updated by Rogers, Chris about 9 years ago

Going through the docs at


  • yum install xml-commons-apis.x86_64 tomcat5 postgresql-jdbc

Nothing done

  • ln -s /usr/share/java/postgresql-jdbc.jar /var/lib/tomcat5/common/lib/postgresql-jdbc.jar

link exists

  • ln -s /usr/share/java/xalan-j2.jar /var/lib/tomcat5/common/lib/xalan-j2.jar

link exists

  • emacs /etc/tomcat5/server.xml; port 8080 -> 4443

ERROR there was no reference to 4443 in the list

  • copy the war file to /var/lib/tomcat5/webapps/

There is a file cdb.war at this location

  • copy the file to /opt/mice/etc/cdb-server/

There is log4j file at this location; note it has been edited

  • configure Configuration DB


ERROR db.url=jdbc:postgresql://localhost/
ERROR ls -lah cdb.props yields
-rw-r--r-- 1 root root 160 Feb 6 13:39 cdb.props

  • create the Configuration DB log file directory

directory exists with correct permissions

  • customise logging levels - log level line is commented, I could uncomment
  • get Tomcat up and running as a service

/sbin/service tomcat5 status
/etc/init.d/tomcat5 is already running (2812)

  • configure Apache to forward messages to Tomcat

I don't know what this means

Attempted Fix

I copied



I edited the file


to change the port for http from 8080 to 4443

I restarted tomcat

/sbin/service/tomcat5 restart

Now I get from


The requested URL could not be retrieved

While trying to retrieve the URL:

The following error was encountered:

    Connection to Failed 

The system returned:

    (111) Connection refused

The remote host or network may be down. Please try the request again.

Your cache administrator is
Generated on Thu, 11 Jul 2013 16:55:17 +0100

I restart tomcat

/sbin/service/tomcat5 restart

and I get back to the other error message

HTTP Status 404 - /cdb/beamline

type Status report

message /cdb/beamline

description The requested resource (/cdb/beamline) is not available.
Apache Tomcat/5.5.23

So I believe tomcat is available and listening on port 8080


Updated by Rogers, Chris about 9 years ago

I also tried adding a port to




each time followed by tomcat restart; neither thing helped.


Updated by Rogers, Chris about 9 years ago

I have to go home. I am on the server running commands like from:

trying to figure out how to list tomcat web applications to investigate the layer between tomcat and postgres.


Updated by Rogers, Chris about 9 years ago

Also from various emails running in parallel

The error is

HTTP Status 404 - /cdb/beamline.wsdl

type Status report

message /cdb/beamline.wsdl

description The requested resource (/cdb/beamline.wsdl) is not available.
Apache Tomcat/5.5.23

where the last line indicates to me that this error message is produced by Tomcat; so server knows something should be presented there but somehow there is a problem in the backend.

On server side I see a log file /var/log/tomcat5/catalina.out indicating something really wrong in the wsdl, below. I believe this means the SOAP layer is failing. If I try a wget from onboard the server (http_proxy="" wget I get a 404. This seems to be what it is tripping on.

10-Jul-2013 16:05:31 org.apache.catalina.startup.Catalina start
INFO: Server startup in 2079 ms
mice.cdb.client.CDBException: Unable to create Beamline. Failed to access the WSDL at: It failed with:
        at mice.cdb.client.beamline.Beamline.init(
        at mice.cdb.client.beamline.Beamline.<init>(
        at mice.cdb.gwt.server.SearchServiceImpl.<init>(
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
        at java.lang.reflect.Constructor.newInstance(
        at java.lang.Class.newInstance0(
        at java.lang.Class.newInstance(
        at org.apache.catalina.core.StandardWrapper.loadServlet(
        at org.apache.catalina.core.StandardWrapper.allocate(
        at org.apache.catalina.core.StandardWrapperValve.invoke(
        at org.apache.catalina.core.StandardContextValve.invoke(
        at org.apache.catalina.core.StandardHostValve.invoke(
        at org.apache.catalina.valves.ErrorReportValve.invoke(
        at org.apache.catalina.valves.AccessLogValve.invoke(
        at org.apache.catalina.core.StandardEngineValve.invoke(
        at org.apache.catalina.connector.CoyoteAdapter.service(
        at org.apache.coyote.http11.Http11Processor.process(
        at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(
        at org.apache.tomcat.util.threads.ThreadPool$
Caused by: Failed to access the WSDL at: It failed with:
        at mice.cdb.service.beamline.BeamlineImplService.<init>(
        at mice.cdb.client.beamline.Beamline.init(
        ... 23 more
Caused by:

Updated by Rogers, Chris about 9 years ago

Editing the tomcat configuration stored at /usr/share/tomcat5/conf; I tried adding deployOnStartup to server.xml Host element; restarted tomcat but still was unable to contact Beamline.wsdl. I reverted the change to server.xml


Updated by Rogers, Chris about 9 years ago

I tried installing the manager, using an xml at /usr/share/tomcat5/conf/Catalina/localhost/manager.xml like

<Context privileged="true" antiResourceLocking="false" 
  <Valve className="org.apache.catalina.valves.RemoteAddrValve" 
         allow="127\.0\.0\.1" />

But catalina.out reports

java.lang.IllegalArgumentException: Document base /usr/share/tomcat5/webapps/manager does not exist or is not a readable directory

indeed the directory does not exist. Documents indicate it comes by default so Antony must have removed it?

Edit to add: I notice that on my SL6 box manager is in a separate RPM to the rest of tomcat. Maybe the correct RPM is not installed?


Updated by Rogers, Chris about 9 years ago

Pierrick sensibly wants some confirmation that the cdb is running in postgres. A little googling led me to

bash-3.2$ psql --username=postgres
psql (9.0.13)
Type "help" for help.

postgres=# \list
                                  List of databases
   Name    |   Owner    | Encoding | Collation  |   Ctype    |   Access privileges   
 cdb       | supermouse | UTF8     | en_GB.utf8 | en_GB.utf8 | 
 postgres  | postgres   | UTF8     | en_GB.utf8 | en_GB.utf8 | 
 template0 | postgres   | UTF8     | en_GB.utf8 | en_GB.utf8 | =c/postgres          +
           |            |          |            |            | postgres=CTc/postgres
 template1 | postgres   | UTF8     | en_GB.utf8 | en_GB.utf8 | postgres=CTc/postgres+
           |            |          |            |            | =c/postgres

I think this means postgres portion of cdb is running.


Updated by Rogers, Chris about 9 years ago

-bash-3.2$ sudo su postgres
[sudo] password for cr67: 
bash-3.2$ psql
psql (9.0.13)
Type "help" for help.

postgres=# \c cdb
You are now connected to database "cdb".
cdb=# \d
                      List of relations
 Schema |            Name            |   Type   |   Owner    
 public | alarm                      | table    | supermouse
 public | alarmhistory               | table    | supermouse
 public | beamline                   | table    | supermouse
 public | beamlinetag                | table    | supermouse
 public | beamlinetag_id_seq         | sequence | supermouse
 public | beamlinetagged             | table    | supermouse
 public | cablingcontrol             | table    | supermouse
 public | cablingdetector            | table    | supermouse
 public | cablingdetector_id_seq     | sequence | supermouse
 public | cablingtracker             | table    | supermouse
 public | calibrationdetector        | table    | supermouse
 public | calibrationdetector_id_seq | sequence | supermouse
 public | calibrationtracker         | table    | supermouse
 public | channels                   | table    | supermouse
 public | controls                   | table    | supermouse
 public | coolingchannel             | table    | supermouse
 public | coolingchannel_id_seq      | sequence | supermouse
 public | coolingchannelcoil         | table    | supermouse
 public | coolingchannelcoiltagged   | table    | supermouse
 public | coolingchannelmagnet       | table    | supermouse
 public | coolingchannelmagnettagged | table    | supermouse
 public | coolingchanneltag          | table    | supermouse
 public | coolingchanneltag_id_seq   | sequence | supermouse
 public | currentstate               | table    | supermouse
 public | daq                        | table    | supermouse
 public | geometry                   | table    | supermouse
 public | geometry_id_seq            | sequence | supermouse
 public | isis                       | table    | supermouse
 public | isisbeam                   | table    | supermouse
 public | ldchosts                   | table    | supermouse
 public | magnet                     | table    | supermouse
 public | magnettagged               | table    | supermouse
 public | pvdata                     | table    | supermouse
 public | run                        | table    | supermouse
 public | runno                      | table    | postgres
 public | scalar                     | table    | supermouse
 public | statemachine               | table    | supermouse
 public | target                     | table    | supermouse
 public | targetblob                 | table    | supermouse
 public | targetfloat                | table    | supermouse
 public | targetinteger              | table    | supermouse
 public | targetold                  | table    | supermouse
 public | targetsetbcd               | table    | supermouse
 public | targetstring               | table    | supermouse
 public | targetuserdelay            | table    | supermouse
(45 rows)

cdb=# select * from alarm;

produces long list of tables


Updated by Rogers, Chris about 9 years ago

Progress - I was able to get a read only copy running by wholesale copying the webapps directory from across to preprodcdb. I noticed a difference

-bash-3.2$ ls /var/lib/tomcat5/webapps/cdb/
doc  index.html  META-INF  WEB-INF
-bash-3.2$ ls /var/lib/tomcat5/webapps/cdb/*-INF/

lib  LICENSE  sun-jaxws.xml  web.xml

whereas for the old webapps directory (which I moved to _webapps)

-bash-3.2$ ls /var/lib/tomcat5/_webapps/cdb/
doc  index.html

I can start digging into these *-INF directories in the morning...


Updated by Rogers, Chris about 9 years ago

  • Status changed from Open to Closed
  • Assignee changed from Martyniak, Janusz to Rogers, Chris
  • % Done changed from 0 to 100

Let me just finish this off - I moved the cdb directory from the production cdb to the preprod cdb and edited the xml files to add in SuperMouse elements to the schema.

The old webapps is saved as /var/lib/tomcat5/webapps_preprod and a copy of production cdb is saved in /var/lib/tomcat5/webapps_preprod

It all seems to be working and no one has complained so I will close the issue.

Also available in: Atom PDF