Data truncation: Data too long for column '

Data truncation: Data too long for column '_stanzaId' (#69)

Wojciech Kapcia (Tigase) opened 5 years ago

ID: YWE4NjAyZWYtOTdkYS00YzJiLWE4OTUtNTlmMjQ5MjI2YWUzYzU1ZjI5YWRiM2E0NjFmYjMzMjI4ODk0NzEyODMwMGMzZmUwYmNkNDlhNDgxMDZmZDgzMDdkYmRlMTQ4MjdkZA==

com.mysql.cj.jdbc.exceptions.MysqlDataTruncation: Data truncation: Data too long for column &apos;_stanzaId&apos; at row 1
	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:104)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:955)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1094)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1042)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeLargeUpdate(ClientPreparedStatement.java:1345)
	at com.mysql.cj.jdbc.CallableStatement.executeLargeUpdate(CallableStatement.java:2526)
	at com.mysql.cj.jdbc.CallableStatement.executeUpdate(CallableStatement.java:896)
	at jdk.internal.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at tigase.db.jdbc.PreparedStatementInvocationHandler.invoke(PreparedStatementInvocationHandler.java:38)
	at com.sun.proxy.$Proxy38.executeUpdate(Unknown Source)
	at tigase.archive.db.JDBCMessageArchiveRepository.archiveMessage(JDBCMessageArchiveRepository.java:325)
	at tigase.archive.db.JDBCMessageArchiveRepository.archiveMessage(JDBCMessageArchiveRepository.java:50)
	at tigase.archive.db.AbstractMessageArchiveRepository.archiveMessage(AbstractMessageArchiveRepository.java:94)
	at tigase.archive.db.JDBCMessageArchiveRepository.archiveMessage(JDBCMessageArchiveRepository.java:121)
	at jdk.internal.reflect.GeneratedMethodAccessor108.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at tigase.stats.StatisticsInvocationHandler.invoke(StatisticsInvocationHandler.java:75)
	at com.sun.proxy.$Proxy58.archiveMessage(Unknown Source)
	at tigase.archive.db.MessageArchiveRepositoryPool.archiveMessage(MessageArchiveRepositoryPool.java:66)
	at tigase.archive.modules.ArchivingModule.process(ArchivingModule.java:101)
	at tigase.component.modules.StanzaProcessor.process(StanzaProcessor.java:164)
	at tigase.component.modules.StanzaProcessor.processPacket(StanzaProcessor.java:85)
	at tigase.component.AbstractKernelBasedComponent.processPacket(AbstractKernelBasedComponent.java:81)
	at tigase.server.AbstractMessageReceiver$QueueListener.run(AbstractMessageReceiver.java:1397)

Activities

Wojciech Kapcia (Tigase) commented 5 years ago

Log entry:

[2020-10-04 15:07:21:732] [WARNING ] [  in_0-message-archive ] JDBCMessageArchiveRepository.archiveMessage(): Problem adding new entry to DB: [message to="…@tigase.im/…" id="YWE4NjAyZWYtOTdkYS00YzJiLWE4OTUtNTlmMjQ5MjI2YWUzYzU1ZjI5YWRiM2E0NjFmYjMzMjI4ODk0NzEyODMwMGMzZmUwYmNkNDlhNDgxMDZmZDgzMDdkYmRlMTQ4MjdkZA==" from="morphix@xmpp.nabla.li/hertzsprung" xmlns="jabber:client"][received id="A4175F6A-E663-401F-97C7-52BC99D2730B" xmlns="urn:xmpp:receipts"/][/message]

Wojciech Kapcia (Tigase) commented 5 years ago

The ID itself is just a Base64 of some UUID (+some other random string). I don't recall specification limiting it in any way, but maybe we should somewhat increase the limit on the field?
Andrzej Wójcik (Tigase) commented 5 years ago

The stanza_id field is a lot larger that UUID and this is non-typical usage of the ID. Increasing size of stanza_id field (which is a varchar due to requirement to index this field) should be small and 64 chars in my opinion is large enough (UUID in string is 36).

I think we should return an error to the sender with type resource-constraint even before item would be sent to the storage. However, stored in asynchronous and message would still be delivered, which is bad.

Wojciech Kapcia (Tigase) commented 5 years ago

After discussing it in the chat: best way forward would be to store null in the database as the field is only used for optimisation - that way we won't loose current functionality and clients that send excessively long IDs would not be handled. Out of consideration for performance we should not increase field length.

Comments from the jdev MUC:

[2020-10-05 13:24:57] <Wojtek>: quick question - should we be concerned with lenght of the stanza id? from what I gather schema doesn't limit it...
[2020-10-05 13:26:03] <pep.>: I think it's profanity that has immensely long IDs. Some kind of HMAC iirc
[2020-10-05 13:26:19] <jonas’>: Wojtek, there is no specified limit beyond the limit of th eoverall stanza
[2020-10-05 13:26:41] <>: I suggest to limit it to 1023 utf-8 bytes (or codepoints, if that’s easier for you) if you have to place a limit
[2020-10-05 13:26:55] <>: it’s the same limit as resource and localpart have
[2020-10-05 13:27:33] <Wojtek>: yeah, but that's not specified, and someone may decide well I wannna have longer IDs. would have been nice to have this in the specs IMHO
[2020-10-05 13:27:48] <>: > it’s the same limit as resource and localpart have
yeah, that seems reasonable
[2020-10-05 13:27:52] <pep.>: Which spec?
[2020-10-05 13:28:06] <>: 6120?
[2020-10-05 13:28:20] <jonas’>: Wojtek, yeah, it would be nice to have a limit written down in '6120

Andrzej Wójcik (Tigase) commented 5 years ago

I've fixed the issue by storing null if stanza id is bigger than 64 chars (our limit). The value is currently not used but was designed to be use only in optimizations like fastening, so not storing that for some stanzas will not make a huge difference.
Login to comment

Type	Bug
Priority	Normal
Assignee	Wojciech Kapcia (Tigase)
Version	tigase-server-8.2.0

Issue Votes (0)

Watchers (0)

Reference

tigase/_server/tigase-message-archiving#69