Projects tigase _server server-core Issues #1265
Optimise DB usage when deleting OfflineMessages in version 8.x (#1265)
Tigase Support opened 3 years ago

please note that on this ticket we do not require/need any help, this ticket is just for you information as requested in the other one. Feel free to use this info and close the ticket.

In attachment you can see the CPU usage of db. The v7 is on db.m5.xlarge with mysql 5.7 the v8 is on db.r5.xlarge with aurora 5.7

during the spike times the CPU of the v8 are much higher than the v7.

the v8 has pub/sub completely disabled

Maybe is the overhead of some lock introduced with the stored procedure on the 8 for adding/deleting messages

Regards Davide

tigase.zip

wojciech.kapcia@tigase.net commented 3 years ago

Thank you for the information. Indeed it looks like offline message processing. We will look into it.

Andrzej Wójcik (Tigase) commented 3 years ago

From version 8.0 database schema is responsible for the calculation of SHA1 of JIDs of sender and recipient of the messages stored in the tig_offline_messages table. This may slightly increase CPU usage and spikes.

However, Tigase in version lower than 8.0 was using many SQL queries to insert a single message, ie. it checked id of a sender JID, checked id of recipient JID, counted no. of stored messages (to check if the limit was exceeded) and inserted a message. Each query was executed as a separate statement issued by Tigase and that:

  • split CPU usage between those queries
  • introduced delays between queries due to I/O delays between Tigase and database.

On version 8.0 and newer all those queries are done in a single stored procedure on the database side reducing I/O delays.

Due to that CPU spikes could be lower on versions lower than 8.0 but would take more time to actually insert a message to the database.

As for removing messages from the database, this now requires the calculation of SHA1 on the database side as well, what if combined with concurrently running statements for inserting messages to offline storage, will increase CPU usage as well (as it may need to wait for those tasks to complete).

After reviewing the database schema, I was able to optimize it by dropping and index, and adjusting the indexed field size, which should reduce the CPU time required to find rows with those indexes and will reduce the size of this index.

wojciech.kapcia@tigase.net commented 3 years ago

Thank you for the explanation and optimisation. Those seem to be ok.

I think that we should apply same changes to all databases.

Andrzej Wójcik (Tigase) commented 3 years ago

I've dropped redundant indexes and adjusted field sizes where needed also for other databases schemas.

issue 1 of 1
Type
Consulting
Priority
Normal
Assignee
Version
Candidate for next minor release
Spent time
15m
Issue Votes (0)
Watchers (0)
Reference
tigase/_server/server-core#1265
Please wait...
Page is in error, reload to recover