StackOverflowError during config dump (#870)

Activities

Andrzej Wójcik (Tigase) opened 7 years ago

Due Date
2017-08-17

As reported by %wojtek Tigase with simple config and custom REST script throws StackOverflowError

admins = [ 'admin@atlantiscity', 'admin@firefly', 'http@pc12.home',
'admin@test-d' ]
'config-type' = '--gen-config-def'

'virtual-hosts' = [ 'atlantiscity', 'firefly', 'test-d', 'test-domain',
'tls-required.com:tls-required' ]
dataSource {
   default {
       uri =
'jdbc:mysql://localhost/tigasedb?user=tigase&password=tigase&useSSL=false'
   }
   'update-stats-upload-data-source' () {
       uri =
'jdbc:mysql://localhost/tigasedb?user=tigase&password=tigase&useSSL=false'
   }
}
http {
   rest {
       'api-keys' = [ 'open_access:regex=/rest/.*' ]
       UpdateChecker () {
           repository () {
               dataSourceName = 'update-stats-upload-data-source'
           }
       }
   }
}
'message-archive' (active: false) {}
muc (active: false) {}
pubsub (active: false) {}

java.lang.StackOverflowError
	at java.util.zip.ZipCoder.getBytes(ZipCoder.java:86)
	at java.util.zip.ZipFile.getEntry(ZipFile.java:310)
	at java.util.jar.JarFile.getEntry(JarFile.java:240)
	at java.util.jar.JarFile.getJarEntry(JarFile.java:223)
	at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1042)
	at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:1020)
	at sun.misc.URLClassPath$1.next(URLClassPath.java:267)
	at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:277)
	at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601)
	at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader$3.next(URLClassLoader.java:598)
	at java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623)
	at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45)
	at
sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54)
	at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45)
	at
sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54)
	at
java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354)
	at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393)
	at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474)
	at javax.xml.transform.FactoryFinder$1.run(FactoryFinder.java:327)
	at java.security.AccessController.doPrivileged(Native Method)
	at
javax.xml.transform.FactoryFinder.findServiceProvider(FactoryFinder.java:323)
	at javax.xml.transform.FactoryFinder.find(FactoryFinder.java:299)
	at
javax.xml.transform.TransformerFactory.newInstance(TransformerFactory.java:106)
	at tigase.pubsub.modules.XsltTool.<init>(XsltTool.java:31)
	at sun.reflect.GeneratedConstructorAccessor147.newInstance(Unknown Source)
	at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
	at
tigase.component.DSLBeanConfigurator.dumpConfigFromSubBeans(DSLBeanConfigurator.java:268)
	at
tigase.component.DSLBeanConfigurator.lambda$dumpConfigFromSubBeans$16(DSLBeanConfigurator.java:354)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1691)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at
tigase.component.DSLBeanConfigurator.dumpConfigFromSubBeans(DSLBeanConfigurator.java:353)
	at
tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:244)
	at
tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:204)
	at
tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:204)
	at
tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:204)
	at
tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:204)
	at
tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:204)

patch.diff

Andrzej Wójcik (Tigase) commented 7 years ago

Wojtek, I've tried to replicate this issue and was not able. I've tried launching Tigase with your config and even with the previous version of UpdateChecker script adjusted to work in Tigase (I had only older not working version) and Tigase started and dumped configuration without any issues.

I suppose that it may have something to do with jars you have in the classpath.

Few questions:

Have you launched Tigase from the command line or from IDE?
Could you make sources of your project available? Or tell me where I can find them?

This way I could recreate the same environment as yours. Or at least make a tar archive of whole Tigase server directory which fails to start so I could unpack it and debug.

Andrzej Wójcik (Tigase) commented 7 years ago

I tried to replicate the issue with commented out constructor of a SD repository bean, but even with that it was working fine. I have no idea, how to replicate and find an issue, without replication of whole environment which you are using.

wojciech.kapcia@tigase.net commented 7 years ago

It happened yesterday a couple of times and today, after using latest server and making a couple of code changes to my code it stopped... I'll keep this open for a while and close it I won't be able to reproduce it.

wojciech.kapcia@tigase.net commented 7 years ago

I did not have to wait long - it just happened! The ST is slightly different, but the gist is the same:

2017-08-08 18:54:20.088 [main]             ThreadExceptionHandler.uncaughtException()  SEVERE: Uncaught thread: "main" exception
java.lang.StackOverflowError
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader$3.next(URLClassLoader.java:598)
	at java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623)
	at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45)
	at sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54)
	at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45)
	at sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54)
	at java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354)
	at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393)
	at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474)
	at javax.xml.transform.FactoryFinder$1.run(FactoryFinder.java:327)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.xml.transform.FactoryFinder.findServiceProvider(FactoryFinder.java:323)
	at javax.xml.transform.FactoryFinder.find(FactoryFinder.java:299)
	at javax.xml.transform.TransformerFactory.newInstance(TransformerFactory.java:106)
	at tigase.pubsub.modules.XsltTool.<init>(XsltTool.java:31)
	at sun.reflect.GeneratedConstructorAccessor146.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
	at tigase.component.DSLBeanConfigurator.dumpConfigFromSubBeans(DSLBeanConfigurator.java:268)
	at tigase.component.DSLBeanConfigurator.lambda$dumpConfigFromSubBeans$16(DSLBeanConfigurator.java:354)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1691)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at tigase.component.DSLBeanConfigurator.dumpConfigFromSubBeans(DSLBeanConfigurator.java:353)
	at tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:244)
	at tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:204)
	at tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:204)
	at tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:204)
	at tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:204)
	at tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:204)
…

However, the first common, Tigase call is the same: at tigase.pubsub.modules.XsltTool.<init>(XsltTool.java:31) (which is a bit weird, as I have @pubsub (active: false) {}@).

I'm using tigase-issue #7.2.0-SNAPSHOT-b4895 with following custom code:

a couple of changes in tigase-server related to Updater and ScheduledTask
custom binary and groovy script for update-checker.

I'm attaching:

modified binaries (to be applied on the mentioned distribution)
diff of the tigase-server sources.

Should be enough. However please note, it was happening for me with normal distributions without my changes.

Andrzej Wójcik (Tigase) commented 7 years ago

With provided jars and sources, I was able to replicate the issue and fix it. The fix is pushed to the repository.

I was not able to recompile server with your patch (missing classes), but I removed only non-compilable parts and still was able to replicate the issue and was able to confirm that fix solves this issue. Please verify with all changes you have that this issue is not happening anymore.

wojciech.kapcia@tigase.net commented 7 years ago

Thank you - seems to be fixed. After using newest version I've got also NPE:

2017-08-09 11:49:32.469 [main]             DSLBeanConfigurator.dumpConfigFromSubBeans()  FINEST: exception retrieving configuration of subbeans = connections
java.lang.NullPointerException
        at tigase.socks5.AbstractConnectionManager$PortsConfigBean.register(AbstractConnectionManager.java:621)
        at tigase.component.DSLBeanConfigurator.dumpConfigFromSubBeans(DSLBeanConfigurator.java:323)
        at tigase.component.DSLBeanConfigurator.lambda$dumpConfigFromSubBeans$16(DSLBeanConfigurator.java:356)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
        at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1691)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
        at tigase.component.DSLBeanConfigurator.dumpConfigFromSubBeans(DSLBeanConfigurator.java:355)
        at tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:246)
        at tigase.component.DSLBeanConfigurator.dumpConfiguration(DSLBeanConfigurator.java:181)
        at tigase.server.Bootstrap.start(Bootstrap.java:173)
        at tigase.server.XMPPServer.start(XMPPServer.java:136)
        at tigase.server.XMPPServer.main(XMPPServer.java:117)

Not sure if it's related or not (though, ST seems to share some parts with the previous one). If not related we can close this.

Andrzej Wójcik (Tigase) commented 7 years ago

It was not related to StackOverflowError but was related to config dump and should not happen, so I've fixed it.

Type	Bug
Priority	Normal
Assignee	wojciech.kapcia@tigase.net
RedmineID	5867
Version	tigase-server-8.0.0
Spent time	20h

Issue Votes (0)

Watchers (0)

Reference

tigase/_server/server-core#870