exception uploading statistics (#836)

Eric Dziewa opened 8 years ago

The new 7.2.0 installation on tpub cannot upload statistics. I have these cronjobs running on tpub, and home on my work computer.

They only ever report a connection error when Tigase (t2, t6) is actually down for a reboot.

curl -X POST --connect-timeout 60 --max-time 120 -sS --data-binary '<test/>' --header 'Content-Type: application/xml' http://192.95.36.80:8080/rest/stats/upload

curl -X POST --connect-timeout 60 --max-time 120 -sS --data-binary '<test/>' --header 'Content-Type: application/xml' http://198.27.120.208:8080/rest/stats/upload

If you view those 2 URLs in your browser they are different. One says Jetty the other does not.

I reverted the installation on tpub to 7.1.0 and the error went away.

However I don't think this is tied to 7.2.0 explicitly. One of our customers has reported this.

They use 7.1, I went to check on cluster-c and the error appears there too.

2017-06-02 16:00:07.472 [Timer-1]          StatisticsUploader$1.run()              SEVERE:   exception uploading statistics
java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
    at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
    at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1283)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1258)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
    at tigase.stats.collector.provider.StatisticsUploader.a(SourceFile:50)
    at tigase.stats.collector.provider.StatisticsUploader$1.run(SourceFile:109)
    at java.util.TimerThread.mainLoop(Timer.java:555)
    at java.util.TimerThread.run(Timer.java:505)

Artur Hefczyc commented 8 years ago

I can see that HTTPS (SSL) is used for connection to upload statistics. So, maybe, the SSL certificate is accepted on some machines and is not accepted on other installations? If the ca-root is not well recognized this could lead to random problems.
Eric Dziewa commented 8 years ago

Now that you've mentioned certificate it gave me an idea. Apache wasn't running on t6. Which is strange because it's in the autostart list. Is working fine now.
Artur Hefczyc commented 8 years ago

Why does it matter? Apache is not used for stats upload, is it?

By the way %Eric , please write some notes while you work on the installation upgrade, anything which you encounter, any issues, doubts, anything which is not straightforward during the installation and upgrade. Then, hand it over to %Daniel. He can then prepare some kind of hints and helpers on 7.2.0 installation and maybe he can also include some of this information in documentation.
Eric Dziewa commented 8 years ago

Apache handles serving the LetsEncrypt cert on port 443. Then acts as a proxt to localhost 8080.
Artur Hefczyc commented 8 years ago

Ok, understand now.
Login to comment

Type	Bug
Priority	Major
Assignee	Artur Hefczyc
RedmineID	5616
Version	tigase-server-8.0.0

Issue Votes (0)

Watchers (0)

Reference

tigase/_server/server-core#836