PreparedStatements optimization (#22)
wojciech.kapcia@tigase.net opened 9 years ago
Due Date
2016-02-23

Current implementation uses a couple of thousands of PreparedStatements for each database connection which may yield huge memory usage.

Andrzej Wójcik (Tigase) commented 9 years ago

%wojtek What do you think about changing access to database in case of Message Archive component from direct execution of generated entries to execution of stored procedures as in other components? I think this way we could create procedures which could accept NULL as some parameters and adjust query on database level, which would limit number of queries need to be kept as stored procedures.

wojciech.kapcia@tigase.net commented 9 years ago

This sounds like a good solution. However while doing that we need to think about handling of #3542 - as far as I know currently it's not possible to have flexible list of parameters but it may be possible to either try to use arrays or slicing Strings in stored procedures - I assume this only should apply to tags/mentions so it should work and give us the flexibility?

Andrzej Wójcik (Tigase) commented 9 years ago

We are not able to use arrays as they are not supported on every database implementation we support, but I suppose that we could use comma separated strings for tags or temporary tables to set them before execution of stored procedure responsible for saving or listing data.

Andrzej Wójcik (Tigase) commented 9 years ago

I created new branch task_3543 with changes for Message Archiving component. As I tested this version is ready and marked as @1.3.0-SNAPSHOT@.

However as I need to adjust code of Unified Archive to make it work with new version of Message Archiving component, I created separate task #3909 in which I will log all informations about progress of refactoring.

I will keep this task in In Progress state until both task will be finished and ready to use.

Andrzej Wójcik (Tigase) commented 9 years ago

I created release branch for version which was in master and merged new changed to master branch to be in Tigase XMPP Server 7.2.0-SNAPSHOT.

There is however no strict dependency on Tigase XMPP Server 7.2.0-SNAPSHOT but I needed to increase this project version number as changes to repository layer were severe.

I updated Jenkins configuration as well.

wojciech.kapcia@tigase.net commented 9 years ago

With connection pool set do 150 connections memory usage dropped from roughly 1G to 115M (using MySQL).

Artur Hefczyc commented 9 years ago

Excellent work! Thanks.

issue 1 of 1
Type
Bug
Priority
Normal
Assignee
RedmineID
3543
Issue Votes (0)
Watchers (0)
Reference
tigase/_server/tigase-message-archiving#22
Please wait...
Page is in error, reload to recover