ClusterConnectionManager Integration (#7)

Eric Dziewa opened 1 decade ago

Due Date
2014-09-16

Blocking: Automated TTS tests on MongoDB

2014-09-10 20:53:39.353 [main]             ClConSQLRepository.initRepository()  WARNING: Problem initializing database:
java.lang.NullPointerException
    at tigase.cluster.repo.ClConSQLRepository.checkDB(ClConSQLRepository.java:308)
    at tigase.cluster.repo.ClConSQLRepository.initRepository(ClConSQLRepository.java:176)
    at tigase.cluster.repo.ClConSQLRepository.setProperties(ClConSQLRepository.java:278)
    at tigase.cluster.ClusterConnectionManager.setProperties(ClusterConnectionManager.java:770)
    at tigase.conf.ConfiguratorAbstract.setup(ConfiguratorAbstract.java:572)
    at tigase.conf.ConfiguratorAbstract.componentAdded(ConfiguratorAbstract.java:183)
    at tigase.conf.Configurator.componentAdded(Configurator.java:50)
    at tigase.conf.Configurator.componentAdded(Configurator.java:33)
    at tigase.server.AbstractComponentRegistrator.addComponent(AbstractComponentRegistrator.java:116)
    at tigase.server.MessageRouter.addComponent(MessageRouter.java:108)
    at tigase.server.MessageRouter.addRouter(MessageRouter.java:145)
    at tigase.server.MessageRouter.setProperties(MessageRouter.java:807)
    at tigase.conf.ConfiguratorAbstract.setup(ConfiguratorAbstract.java:572)
    at tigase.conf.ConfiguratorAbstract.componentAdded(ConfiguratorAbstract.java:183)
    at tigase.conf.Configurator.componentAdded(Configurator.java:50)
    at tigase.conf.Configurator.componentAdded(Configurator.java:33)
    at tigase.server.AbstractComponentRegistrator.addComponent(AbstractComponentRegistrator.java:116)
    at tigase.server.MessageRouter.addRegistrator(MessageRouter.java:131)
    at tigase.server.MessageRouter.setConfig(MessageRouter.java:700)
    at tigase.server.XMPPServer.start(XMPPServer.java:142)
    at tigase.server.XMPPServer.main(XMPPServer.java:112)
2014-09-10 20:53:39.506 [main]             SimpleCache.<init>()               WARNING:  Tigase cache turned off.
2014-09-10 20:53:39.805 [main]             SimpleCache.<init>()               WARNING:  Tigase cache turned off.
2014-09-10 20:53:40.100 [main]             StanzaReceiver.setProperties()     SEVERE:   Can't initialize repository:
tigase.db.DBInitException: Problem initializing jdbc connection: mongodb://localhost/tigasetest
    at tigase.db.jdbc.JDBCRepository.initRepository(JDBCRepository.java:828)
    at tigase.db.RepositoryFactory.getUserRepository(RepositoryFactory.java:660)
    at tigase.server.sreceiver.StanzaReceiver.setProperties(StanzaReceiver.java:668)
    at tigase.conf.ConfiguratorAbstract.setup(ConfiguratorAbstract.java:572)
    at tigase.conf.ConfiguratorAbstract.componentAdded(ConfiguratorAbstract.java:183)
    at tigase.conf.Configurator.componentAdded(Configurator.java:50)
    at tigase.conf.Configurator.componentAdded(Configurator.java:33)
    at tigase.server.AbstractComponentRegistrator.addComponent(AbstractComponentRegistrator.java:116)
    at tigase.server.MessageRouter.addComponent(MessageRouter.java:108)
    at tigase.server.MessageRouter.addRouter(MessageRouter.java:145)
    at tigase.server.MessageRouter.setProperties(MessageRouter.java:807)
    at tigase.conf.ConfiguratorAbstract.setup(ConfiguratorAbstract.java:572)
    at tigase.conf.ConfiguratorAbstract.componentAdded(ConfiguratorAbstract.java:183)
    at tigase.conf.Configurator.componentAdded(Configurator.java:50)
    at tigase.conf.Configurator.componentAdded(Configurator.java:33)
    at tigase.server.AbstractComponentRegistrator.addComponent(AbstractComponentRegistrator.java:116)
    at tigase.server.MessageRouter.addRegistrator(MessageRouter.java:131)
    at tigase.server.MessageRouter.setConfig(MessageRouter.java:700)
    at tigase.server.XMPPServer.start(XMPPServer.java:142)
    at tigase.server.XMPPServer.main(XMPPServer.java:112)
Caused by: java.lang.NullPointerException
    at tigase.db.jdbc.JDBCRepository.checkDBSchema(JDBCRepository.java:1363)
    at tigase.db.jdbc.JDBCRepository.initRepository(JDBCRepository.java:792)
    at tigase.db.RepositoryFactory.getUserRepository(RepositoryFactory.java:660)
    at tigase.server.sreceiver.StanzaReceiver.setProperties(StanzaReceiver.java:668)
    at tigase.conf.ConfiguratorAbstract.setup(ConfiguratorAbstract.java:572)
    at tigase.conf.ConfiguratorAbstract.componentAdded(ConfiguratorAbstract.java:183)
    at tigase.conf.Configurator.componentAdded(Configurator.java:50)
    at tigase.conf.Configurator.componentAdded(Configurator.java:33)
    at tigase.server.AbstractComponentRegistrator.addComponent(AbstractComponentRegistrator.java:116)
    at tigase.server.MessageRouter.addComponent(MessageRouter.java:108)
    at tigase.server.MessageRouter.addRouter(MessageRouter.java:145)
    at tigase.server.MessageRouter.setProperties(MessageRouter.java:807)
    at tigase.conf.ConfiguratorAbstract.setup(ConfiguratorAbstract.java:572)
    at tigase.conf.ConfiguratorAbstract.componentAdded(ConfiguratorAbstract.java:183)
    at tigase.conf.Configurator.componentAdded(Configurator.java:50)
    at tigase.conf.Configurator.componentAdded(Configurator.java:33)
    at tigase.server.AbstractComponentRegistrator.addComponent(AbstractComponentRegistrator.java:116)
    at tigase.server.MessageRouter.addRegistrator(MessageRouter.java:131)
    at tigase.server.MessageRouter.setConfig(MessageRouter.java:700)
    at tigase.server.XMPPServer.start(XMPPServer.java:142)
    at tigase.server.XMPPServer.main(XMPPServer.java:112)
2014-09-10 20:53:40.102 [main]             Configurator.componentAdded()      WARNING:  ignoring NPE
java.lang.NullPointerException
    at tigase.server.sreceiver.StanzaReceiver.saveTaskToRepository(StanzaReceiver.java:829)
    at tigase.server.sreceiver.StanzaReceiver.addTaskInstance(StanzaReceiver.java:748)
    at tigase.server.sreceiver.StanzaReceiver.setProperties(StanzaReceiver.java:695)
    at tigase.conf.ConfiguratorAbstract.setup(ConfiguratorAbstract.java:572)
    at tigase.conf.ConfiguratorAbstract.componentAdded(ConfiguratorAbstract.java:183)
    at tigase.conf.Configurator.componentAdded(Configurator.java:50)
    at tigase.conf.Configurator.componentAdded(Configurator.java:33)
    at tigase.server.AbstractComponentRegistrator.addComponent(AbstractComponentRegistrator.java:116)
    at tigase.server.MessageRouter.addComponent(MessageRouter.java:108)
    at tigase.server.MessageRouter.addRouter(MessageRouter.java:145)
    at tigase.server.MessageRouter.setProperties(MessageRouter.java:807)
    at tigase.conf.ConfiguratorAbstract.setup(ConfiguratorAbstract.java:572)
    at tigase.conf.ConfiguratorAbstract.componentAdded(ConfiguratorAbstract.java:183)
    at tigase.conf.Configurator.componentAdded(Configurator.java:50)
    at tigase.conf.Configurator.componentAdded(Configurator.java:33)
    at tigase.server.AbstractComponentRegistrator.addComponent(AbstractComponentRegistrator.java:116)
    at tigase.server.MessageRouter.addRegistrator(MessageRouter.java:131)
    at tigase.server.MessageRouter.setConfig(MessageRouter.java:700)
    at tigase.server.XMPPServer.start(XMPPServer.java:142)
    at tigase.server.XMPPServer.main(XMPPServer.java:112)
2014-09-10 20:53:40.121 [main]             ConfigurationCache.store()         WARNING:  Dumping server configuration to: etc/config-dump.properties
2014-09-10 20:53:54.353 [cluster-nodes]    ThreadExceptionHandler.uncaughtException()  SEVERE: Uncaught thread: "cluster-nodes" exception
java.lang.ArrayIndexOutOfBoundsException: 0
    at java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:368)
    at java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:377)
    at tigase.db.DataRepositoryPool.takeRepo(DataRepositoryPool.java:80)
    at tigase.db.DataRepositoryPool.getPreparedStatement(DataRepositoryPool.java:176)
    at tigase.db.DataRepositoryPool.getPreparedStatement(DataRepositoryPool.java:179)
    at tigase.cluster.repo.ClConSQLRepository.storeItem(ClConSQLRepository.java:198)
    at tigase.cluster.repo.ClConConfigRepository.reload(ClConConfigRepository.java:97)
    at tigase.cluster.repo.ClConSQLRepository.reload(ClConSQLRepository.java:234)
    at tigase.db.comp.ConfigRepository$1.run(ConfigRepository.java:79)
    at java.util.TimerThread.mainLoop(Timer.java:555)
    at java.util.TimerThread.run(Timer.java:505)

Activities

Andrzej Wójcik (Tigase) commented 1 decade ago

I finished fixing this issue. I needed to implement only few things to add support for MongoDB but I needed to do a lot of refactoring. Source code is ready in repositories and future nightly build should contain this fixes. Fixes are in Tigase XMPP Server project and in Tigase MongoDB project.

Eric Dziewa commented 1 decade ago

Andrzej, I've copied the most recent jars from jenkins: tigase-server.jar, and tigase-issue #1.0.0-SNAPSHOT.jar. Is it okay to drop those into the jars/ directory of b3669 or should I wait for a complete package... are there other dependencies?

2014-09-16 15:59:26.697 [default-threads-pool Queue Worker 1]  WorkerThread.run()  SEVERE: tigase.server.xmppsession.SessionManager$ProcessorWorkerThread,(default-threads-pool Queue Worker 1) Exception during packet processing: from=c2s@build.xmpp-test.net/127.0.0.1_5222_127.0.0.1_34453, to=sess-man@build.xmpp-test.net, DATA=<iq type="set" id="reg2" xmlns="jabber:client"><query xmlns="jabber:iq:register"><username>admin</username><password>stats</password><email>test_user@localhost</email></query></iq>, SIZE=180, XMLNS=jabber:client, PRIORITY=NORMAL, PERMISSION=NONE, TYPE=set
java.lang.ArithmeticException: / by zero
    at tigase.db.DataRepositoryPool.takeRepo(DataRepositoryPool.java:75)
    at tigase.db.DataRepositoryPool.getPreparedStatement(DataRepositoryPool.java:176)
    at tigase.db.DataRepositoryPool.getPreparedStatement(DataRepositoryPool.java:179)
    at tigase.db.jdbc.TigaseCustomAuth.addUser(TigaseCustomAuth.java:347)
    at tigase.db.AuthRepositoryMDImpl.addUser(AuthRepositoryMDImpl.java:82)
    at tigase.xmpp.RepositoryAccess.register(RepositoryAccess.java:473)
    at tigase.xmpp.impl.JabberIqRegister.process(JabberIqRegister.java:312)
    at tigase.server.xmppsession.SessionManager$ProcessorWorkerThread.process(SessionManager.java:2772)
    at tigase.util.WorkerThread.run(WorkerThread.java:132)

Andrzej Wójcik (Tigase) commented 1 decade ago

I think that dropping new versions of Tigase XMPP Server and Tigase MongoDB into old build should work, but I already know that current nightly was broken due to issue in Tigase HTTP API.

This exception which you received I suppose is due to --auth-db entry in etc/init.properties@. It should be omitted to make sure that Tigase XMPP Server will choose correct implementation of @AuthRepository for MongoDB.
Eric Dziewa commented 1 decade ago

Andrzej, Everything seems to be working fine now just the pubsub tests are failing consistently. Also Multi 6, and Privacy 1:

http://graph.cluster-c.xmpp-test.net/5.3.0-SNAPSHOT-b3674/func/mongodb/functional-tests.html

http://graph.cluster-c.xmpp-test.net/5.3.0-SNAPSHOT-b3674/lmem/mongodb/low-memory-tests.html
Andrzej Wójcik (Tigase) commented 1 decade ago
I looked in logs and I have seen that PubSub tests failed due to construction of requests I used when implementing support for MongoDB as I used newest stable version of MongoDB and new features introduced in it. In tests MongoDB 2.0.4 was used while I was developing support using MongoDB 2.6.3.

I suppose we can force usage of MongoDB at least in 2.6.3 version as it is stable now and we are adding support for MongoDB as a new feature to upcoming version. So I suppose we should rerun this tests using MongoDB 2.6.3. I will add a note on Tigase MongoDB project which states that to use it we require MongoDB at least in version 2.6.3 (my bad that I did not added this info before).

Moreover I think we have issue with Privacy tests as they send

<iq id="privacy_9" xmlns="jabber:client" from="multi-reg_5_3@test-d/def-user-resource" type="set"><query xmlns="jabber:iq:privacy"><default/></query></iq>

while according to current code of Tigase XMPP Server <default/> element should contain name of list as CData (value of element).

%kobit - could you check if I'm right about this issue with Privacy tests?
Eric Dziewa commented 1 decade ago

Andrzej, I upgraged mongodb to 2.6.4. That fixed the pubsub tests. Multi 6, and Privacy 1 are still failing. The URLs have been updated.
Andrzej Wójcik (Tigase) commented 1 decade ago

Multi 6 and Privacy 1 are using Privacy plugin and this errors are caused by (I think) error in Privacy test.

Artur, could you look at my previous comment about issue in Privacy test and could you comment on this?
Artur Hefczyc commented 1 decade ago
I do not see a problem with requiring MongoDB version 2.6.3 or later.

I am assigning this to Wojciech as he worked on the privacy lists some time ago. It is very likely the Tigase code was updated and tests were not.

Wojciech, please review the issue with privacy lists.
Wojciech Kapcia (Tigase) commented 1 decade ago

Artur Hefczyc wrote:

I am assigning this to Wojciech as he worked on the privacy lists some time ago. It is very likely the Tigase code was updated and tests were not.

Wojciech, please review the issue with privacy lists.

Those are the same tests run against other DBs and the tests pass just fine, see any results from http://build.tigase.org/nightlies/tests/, hence it seems issue is not related to Privacy Lists code or test run against it.

while according to current code of Tigase XMPP Server element should contain name of list as CData (value of element).

Actually it's not - it follows specs and name of the default list should be taken from name attribute of <default/> element - not CData, but this attribute is optional and if such is not present then client simply declines use of default list.

After digging deeper it looks like there may be a problem with tigase.db.UserRepository.getSubnodes() method in MongoDB repository implementation.
Wojciech Kapcia (Tigase) commented 1 decade ago

I've pushed one change to the Privacy class with additional check, however it's possible that there is still issue with MongoDB integration.
Eric Dziewa commented 1 decade ago

That worked. All tests have completed successful.
Andrzej Wójcik (Tigase) commented 1 decade ago

I changed implementation of getSubnodes() for MongoDB to return null if no subnodes exists as in UserRepository implementation for JDBC instead of empty array which was returned before and caused issue with Privacy class.
Login to comment

Type	Task
Priority	Normal
Assignee	Wojciech Kapcia (Tigase)
RedmineID	2252

Issue Votes (0)

Watchers (0)

Reference

tigase/_server/tigase-mongodb#7

I am assigning this to Wojciech as he worked on the privacy lists some time ago. It is very likely the Tigase code was updated and tests were not.