SSLProtocolException: Input record too big: max = 16709 len = 19190 (#1361)

wojciech.kapcia@tigase.net opened 1 year ago

[2023-06-26 15:57:05:159] [INFO    ] [      pool-11-thread-1 ] LicenceChecker.b()               : Licence OK
javax.net.ssl.SSLProtocolException: Input record too big: max = 16709 len = 19190
	at java.base/sun.security.ssl.SSLEngineImpl.readRecord(Unknown Source)
	at java.base/sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
	at java.base/sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
	at java.base/javax.net.ssl.SSLEngine.unwrap(Unknown Source)
	at tigase.io.JcaTLSWrapper.unwrap(JcaTLSWrapper.java:279)
	at tigase.io.TLSIO.decodeData(TLSIO.java:349)
	at tigase.io.TLSIO.processHandshake(TLSIO.java:327)
	at tigase.server.xmppclient.C2SIOService.extractTlsHandshakeData(C2SIOService.java:182)
	at tigase.server.xmppclient.C2SIOService.handleMalformedInput(C2SIOService.java:152)
	at tigase.net.IOService.readData(IOService.java:809)
	at tigase.xmpp.XMPPIOService.processSocketData(XMPPIOService.java:486)
	at tigase.net.IOService.call(IOService.java:212)
	at tigase.xmpp.XMPPIOService.call(XMPPIOService.java:155)
	at tigase.xmpp.XMPPIOService.call(XMPPIOService.java:54)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
javax.net.ssl.SSLProtocolException: Input record too big: max = 16709 len = 57065
	at java.base/sun.security.ssl.SSLEngineImpl.readRecord(Unknown Source)
	at java.base/sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
	at java.base/sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
	at java.base/javax.net.ssl.SSLEngine.unwrap(Unknown Source)
	at tigase.io.JcaTLSWrapper.unwrap(JcaTLSWrapper.java:279)
	at tigase.io.TLSIO.decodeData(TLSIO.java:349)
	at tigase.io.TLSIO.processHandshake(TLSIO.java:327)
	at tigase.server.xmppclient.C2SIOService.extractTlsHandshakeData(C2SIOService.java:182)
	at tigase.server.xmppclient.C2SIOService.handleMalformedInput(C2SIOService.java:152)
	at tigase.net.IOService.readData(IOService.java:809)
	at tigase.xmpp.XMPPIOService.processSocketData(XMPPIOService.java:486)
	at tigase.net.IOService.call(IOService.java:212)
	at tigase.xmpp.XMPPIOService.call(XMPPIOService.java:155)
	at tigase.xmpp.XMPPIOService.call(XMPPIOService.java:54)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

Referenced from commit 10 months ago

Feature #1361 : Account registration limits

Bartosz Małkowski committed 9 years ago

468eaccf

Andrzej Wójcik (Tigase) changed fields 10 months ago

Name	Previous Value	Current Value
Assignee	wojtek	wojtek, andrzej.wojcik

Andrzej Wójcik (Tigase) commented 10 months ago

As this issue is more common now (at least a few each day) and I've introduced C2SIOService.handleMalformedInput(C2SIOService.java:152) method, I took a look on this issue.

This method is responsible for handling data sent after <starttls/> by the client, ie. when client sends in a single TCP frame <starttls/> followed by SSL/TLS handshake packet. After a detailed review, I've found out that all issues/exceptions are reported from line 152 of C2SIOService.handleMalformedInput(). This call is done only for handling old SSLv2 Client Hello.

Currently, most of the clients do now use SSLv2 Client Hello (nor they use SSLv2 at all), but some old software still may be using this handshake for compatibility reasons. (This handshake may be used to establish even nerver SSLv3 or even TLSv1.3).

It looks like, when in the "plain" or already TLS encrypted connection happens issue with UTF-8 malformation, Tigase checks if that is not a handshake for TLS (initial packet) by checking TLS/SSL packet header. For SSv2 test was very simple and taking into account possible content that is parsed it could cause false positives and cause SSL handshake to be executed (or rather plaintext data was processed by SSLEngine once again causing issue).

To fix that, I've increased logging of data in extractTlsHandshakeData() method to allow us check what actually was fed to SSL engine and improved verification of SSLv3 handshake header (checked if length of the header does not exceed 16709 bytes).

I think that this fix will be OK. If not, we may want to consider dropping support for SSLv2 Client Hello.

Referenced from commit 10 months ago

Improved handling initial handshake SSL packet after processing <starttls/> #1361

Andrzej Wójcik (Tigase) committed 10 months ago

c95a259b

Andrzej Wójcik (Tigase) changed state to 'In Progress' 10 months ago

Previous Value	Current Value
Open	In Progress

Andrzej Wójcik (Tigase) changed state to 'In QA' 10 months ago

Previous Value	Current Value
In Progress	In QA

Referenced from commit 6 months ago

Reducing logging level for handling malformed input similar to TLS handshake #1361

Andrzej Wójcik (Tigase) committed 6 months ago

f1593887

wojciech.kapcia@tigase.net commented 5 months ago

I assume this is completed and the warnings from our installations are just due version with decreased logging not deployed yet and this can be closed?

Andrzej Wójcik (Tigase) commented 5 months ago

Yes, you are correct. Version with this change was not deployed at sure.im/tigase.im.

wojciech.kapcia@tigase.net batch edited 4 months ago

Name	Previous Value	Current Value
Iterations	empty	Candidate for next minor release

wojciech.kapcia@tigase.net changed fields 3 months ago

Name	Previous Value	Current Value
Version	Candidate for next minor release	tigase-server-8.4.0

wojciech.kapcia@tigase.net changed state to 'Closed' 3 months ago

Previous Value	Current Value
In QA	Closed

wojciech.kapcia@tigase.net added to iteration "tigase-server-8.4.0" 3 months ago

wojciech.kapcia@tigase.net removed from iteration "Candidate for next minor release" 3 months ago

Type	Bug
Priority	Normal
Assignee	Andrzej Wójcik (Tigase) wojciech.kapcia@tigase.net
Version	tigase-server-8.4.0