Bug #1306
preprodcb non communicative
100%
Description
See notes at:
http://micewww.pp.rl.ac.uk/projects/computing-software/wiki/PC_20130710
and:
Doing a little investigation - it does not appear to be a firewall
issue. I can contact preprodcdb on port 80 both inside and outside the
firewall (using a thing Antony showed me a couple of weeks ago).
When working outside the firewall, I use data connection on my mobile
phone as an internet access point (to prove I am really outside the
firewall).
> $ nmap preprodcdb.mice.rl.ac.uk -p80 Starting Nmap 5.21 ( http://nmap.org ) at 2013-07-09 09:07 BST Nmap scan report for preprodcdb.mice.rl.ac.uk (130.246.92.153) Host is up (0.0023s latency). rDNS record for 130.246.92.153: heplnv153.pp.rl.ac.uk PORT STATE SERVICE 80/tcp open http Nmap done: 1 IP address (1 host up) scanned in 0.04 seconds
However, when I try to contact preprod using the cdb python library I
get an error, regardless of whether I am inside the firewall or not
> $ python >>> from cdb import Beamline >>> bl = Beamline('http://preprodcdb.mice.rl.ac.uk') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/cr67/MAUS/maus_merge/third_party/install/lib/python2.7/site-packages/cdb/_beamline.py", line 45, in __init__ super(Beamline, self).__init__(url, wsdl_dir) File "/home/cr67/MAUS/maus_merge/third_party/install/lib/python2.7/site-packages/cdb/_base.py", line 39, in __init__ _cdb_wsdl) cdb._exceptions.CdbTemporaryError: Unable to contact CDB server at http://preprodcdb.mice.rl.ac.uk/cdb/beamline?wsdl
I believe this indicates a problem with the wsdl service on the
server, not the firewall. Not sure if that helps, more locates who to
blame.
Updated by Rogers, Chris over 10 years ago
cdb.log indicates last access was at
'2013-06-28 17:27:18.197', 'SS2', 'COLD_READY'); 2013-06-28 17:27:18,332 [http-8080-Processor25] DEBUG mice.cdb.service.DBAccess - Connection closed
Updated by Rogers, Chris over 10 years ago
Going through the docs at
http://micewww.pp.rl.ac.uk/projects/configdb/wiki/PublicInterface
--
yum install xml-commons-apis.x86_64 tomcat5 postgresql-jdbc
Nothing done
ln -s /usr/share/java/postgresql-jdbc.jar /var/lib/tomcat5/common/lib/postgresql-jdbc.jar
link exists
ln -s /usr/share/java/xalan-j2.jar /var/lib/tomcat5/common/lib/xalan-j2.jar
link exists
emacs /etc/tomcat5/server.xml; port 8080 -> 4443
ERROR there was no reference to 4443 in the list
copy the war file to /var/lib/tomcat5/webapps/
There is a file cdb.war at this location
copy the log4j.properties file to /opt/mice/etc/cdb-server/
There is log4j file at this location; note it has been edited
- configure Configuration DB
/opt/mice/etc/cdb-server/cdb.props
ERROR db.url=jdbc:postgresql://localhost/
ERROR ls -lah cdb.props
yields-rw-r--r-- 1 root root 160 Feb 6 13:39 cdb.props
- create the Configuration DB log file directory
directory exists with correct permissions
- customise logging levels - log level line is commented, I could uncomment
- get Tomcat up and running as a service
/sbin/service tomcat5 status
/etc/init.d/tomcat5 is already running (2812)
- configure Apache to forward messages to Tomcat
I don't know what this means
Attempted Fix¶
I copied /etc/tomcat5/server.xml
to
/etc/tomcat5/server.xml.bak
I edited the file
/etc/tomcat5/server.xml
to change the port for http from 8080 to 4443
I restarted tomcat
/sbin/service/tomcat5 restart
Now I get from
http://preprodcdb.mice.rl.ac.uk/cdb/beamline?wsdl
ERROR The requested URL could not be retrieved While trying to retrieve the URL: http://preprodcdb.mice.rl.ac.uk/cdb/beamline?wsdl The following error was encountered: Connection to 130.246.92.153 Failed The system returned: (111) Connection refused The remote host or network may be down. Please try the request again. Your cache administrator is netsdl@stfc.ac.uk. Generated on Thu, 11 Jul 2013 16:55:17 +0100
I restart tomcat
/sbin/service/tomcat5 restart
and I get back to the other error message
HTTP Status 404 - /cdb/beamline type Status report message /cdb/beamline description The requested resource (/cdb/beamline) is not available. Apache Tomcat/5.5.23
So I believe tomcat is available and listening on port 8080
Updated by Rogers, Chris over 10 years ago
I also tried adding a port to
/opt/mice/etc/cdb-server/cdb.props
both
db.url=jdbc:postgresql://localhost:8080/
db.url=jdbc:postgresql://localhost:5432/
each time followed by tomcat restart; neither thing helped.
Updated by Rogers, Chris over 10 years ago
I have to go home. I am on the server running commands like from:
http://tomcat.apache.org/tomcat-6.0-doc/manager-howto.html#Configuring_Manager_Application_Access
trying to figure out how to list tomcat web applications to investigate the layer between tomcat and postgres.
Updated by Rogers, Chris over 10 years ago
Also from various emails running in parallel
The error is HTTP Status 404 - /cdb/beamline.wsdl type Status report message /cdb/beamline.wsdl description The requested resource (/cdb/beamline.wsdl) is not available. Apache Tomcat/5.5.23 where the last line indicates to me that this error message is produced by Tomcat; so server knows something should be presented there but somehow there is a problem in the backend. On server side I see a log file /var/log/tomcat5/catalina.out indicating something really wrong in the wsdl, below. I believe this means the SOAP layer is failing. If I try a wget from onboard the server (http_proxy="" wget http://127.0.0.1/cdb/beamline.wsdl) I get a 404. This seems to be what it is tripping on. 10-Jul-2013 16:05:31 org.apache.catalina.startup.Catalina start INFO: Server startup in 2079 ms mice.cdb.client.CDBException: Unable to create Beamline. Failed to access the WSDL at: http://127.0.0.1:8080/cdb/beamline?wsdl. It failed with: http://127.0.0.1:8080/cdb/beamline?wsdl. at mice.cdb.client.beamline.Beamline.init(Beamline.java:90) at mice.cdb.client.beamline.Beamline.<init>(Beamline.java:68) at mice.cdb.gwt.server.SearchServiceImpl.<init>(SearchServiceImpl.java:35) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:532) at java.lang.Class.newInstance0(Class.java:374) at java.lang.Class.newInstance(Class.java:327) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1089) at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:791) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:127) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:542) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:679) Caused by: javax.xml.ws.WebServiceException: Failed to access the WSDL at: http://127.0.0.1:8080/cdb/beamline?wsdl. It failed with: http://127.0.0.1:8080/cdb/beamline?wsdl. at com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.tryWithMex(RuntimeWSDLParser.java:151) at com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:133) at com.sun.xml.internal.ws.client.WSServiceDelegate.parseWSDL(WSServiceDelegate.java:254) at com.sun.xml.internal.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:217) at com.sun.xml.internal.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:165) at com.sun.xml.internal.ws.spi.ProviderImpl.createServiceDelegate(ProviderImpl.java:93) at javax.xml.ws.Service.<init>(Service.java:76) at mice.cdb.service.beamline.BeamlineImplService.<init>(BeamlineImplService.java:42) at mice.cdb.client.beamline.Beamline.init(Beamline.java:87) ... 23 more Caused by: java.io.FileNotFoundException: http://127.0.0.1:8080/cdb/beamline?wsdl at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1401) at java.net.URL.openStream(URL.java:1031) at com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.createReader(RuntimeWSDLParser.java:793) at com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.resolveWSDL(RuntimeWSDLParser.java:251) at com.sun.xml.internal.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:118)
Updated by Rogers, Chris over 10 years ago
Editing the tomcat configuration stored at /usr/share/tomcat5/conf
; I tried adding deployOnStartup to server.xml Host element; restarted tomcat but still was unable to contact Beamline.wsdl. I reverted the change to server.xml
Updated by Rogers, Chris over 10 years ago
I tried installing the manager, using an xml at /usr/share/tomcat5/conf/Catalina/localhost/manager.xml
like
<Context privileged="true" antiResourceLocking="false" docBase="/usr/share/tomcat5/webapps/manager"> <Valve className="org.apache.catalina.valves.RemoteAddrValve" allow="127\.0\.0\.1" /> </Context>
But catalina.out reports
java.lang.IllegalArgumentException: Document base /usr/share/tomcat5/webapps/manager does not exist or is not a readable directory
indeed the directory does not exist. Documents indicate it comes by default so Antony must have removed it?
Edit to add: I notice that on my SL6 box manager is in a separate RPM to the rest of tomcat. Maybe the correct RPM is not installed?
Updated by Rogers, Chris over 10 years ago
Pierrick sensibly wants some confirmation that the cdb is running in postgres. A little googling led me to
bash-3.2$ psql --username=postgres psql (9.0.13) Type "help" for help. postgres=# \list List of databases Name | Owner | Encoding | Collation | Ctype | Access privileges -----------+------------+----------+------------+------------+----------------------- cdb | supermouse | UTF8 | en_GB.utf8 | en_GB.utf8 | postgres | postgres | UTF8 | en_GB.utf8 | en_GB.utf8 | template0 | postgres | UTF8 | en_GB.utf8 | en_GB.utf8 | =c/postgres + | | | | | postgres=CTc/postgres template1 | postgres | UTF8 | en_GB.utf8 | en_GB.utf8 | postgres=CTc/postgres+ | | | | | =c/postgres
I think this means postgres portion of cdb is running.
Updated by Rogers, Chris over 10 years ago
-bash-3.2$ sudo su postgres [sudo] password for cr67: bash-3.2$ psql psql (9.0.13) Type "help" for help. postgres=# \c cdb You are now connected to database "cdb". cdb=# \d List of relations Schema | Name | Type | Owner --------+----------------------------+----------+------------ public | alarm | table | supermouse public | alarmhistory | table | supermouse public | beamline | table | supermouse public | beamlinetag | table | supermouse public | beamlinetag_id_seq | sequence | supermouse public | beamlinetagged | table | supermouse public | cablingcontrol | table | supermouse public | cablingdetector | table | supermouse public | cablingdetector_id_seq | sequence | supermouse public | cablingtracker | table | supermouse public | calibrationdetector | table | supermouse public | calibrationdetector_id_seq | sequence | supermouse public | calibrationtracker | table | supermouse public | channels | table | supermouse public | controls | table | supermouse public | coolingchannel | table | supermouse public | coolingchannel_id_seq | sequence | supermouse public | coolingchannelcoil | table | supermouse public | coolingchannelcoiltagged | table | supermouse public | coolingchannelmagnet | table | supermouse public | coolingchannelmagnettagged | table | supermouse public | coolingchanneltag | table | supermouse public | coolingchanneltag_id_seq | sequence | supermouse public | currentstate | table | supermouse public | daq | table | supermouse public | geometry | table | supermouse public | geometry_id_seq | sequence | supermouse public | isis | table | supermouse public | isisbeam | table | supermouse public | ldchosts | table | supermouse public | magnet | table | supermouse public | magnettagged | table | supermouse public | pvdata | table | supermouse public | run | table | supermouse public | runno | table | postgres public | scalar | table | supermouse public | statemachine | table | supermouse public | target | table | supermouse public | targetblob | table | supermouse public | targetfloat | table | supermouse public | targetinteger | table | supermouse public | targetold | table | supermouse public | targetsetbcd | table | supermouse public | targetstring | table | supermouse public | targetuserdelay | table | supermouse (45 rows) cdb=# select * from alarm;
produces long list of tables
Updated by Rogers, Chris over 10 years ago
Progress - I was able to get a read only copy running by wholesale copying the webapps directory from cdb.mice.rl.ac.uk across to preprodcdb. I noticed a difference
-bash-3.2$ ls /var/lib/tomcat5/webapps/cdb/ doc index.html META-INF WEB-INF -bash-3.2$ ls /var/lib/tomcat5/webapps/cdb/*-INF/ /var/lib/tomcat5/webapps/cdb/META-INF/: MANIFEST.MF /var/lib/tomcat5/webapps/cdb/WEB-INF/: lib LICENSE sun-jaxws.xml web.xml
whereas for the old webapps directory (which I moved to _webapps)
-bash-3.2$ ls /var/lib/tomcat5/_webapps/cdb/ doc index.html
I can start digging into these *-INF directories in the morning...
Updated by Rogers, Chris over 10 years ago
- Status changed from Open to Closed
- Assignee changed from Martyniak, Janusz to Rogers, Chris
- % Done changed from 0 to 100
Let me just finish this off - I moved the cdb directory from the production cdb to the preprod cdb and edited the xml files to add in SuperMouse elements to the schema.
The old webapps is saved as /var/lib/tomcat5/webapps_preprod
and a copy of production cdb is saved in /var/lib/tomcat5/webapps_preprod
It all seems to be working and no one has complained so I will close the issue.