Projects tigase _server server-core Issues #632
Better defaults (#632)
Closed
wojciech.kapcia@tigase.net opened 9 years ago
Due Date
2016-03-08
  • increase number of DB connections of make it CPU bound;

  • increase number of plugins processing threads or make it CPU bound;

  • add option to configure the baseline or the multiplication factor of the base value.

Artur Hefczyc commented 9 years ago

Unless I do not understand it correctly, this is all done long time ago. Right now, the number of threads per plugin is calculated to the number of CPU cores by default. There is an API for a plugin to override the default and some IO/DB bound plugins make use of the API assigning more threads for themselves - concurrentQueuesNo().

There is also a configuration option to fine tune the number, which can be adjusted for each plugin individually: http://docs.tigase.org/tigase-server/snapshot/Properties_Guide/webhelp/smPlugins.html

wojciech.kapcia@tigase.net commented 9 years ago

This is related in a way to #3307. Yes, number of threads is based on the number of CPU cores (I wasn't clear enough) and it's possible to increase this number but this is on per-plugin base (which may be a bit of hassle if we want to do that for lot of plugins), so the proposal was to allow either providing new base (i.e. @--sm-plugins-thread-base=10@) or multiplication factor so instead of providing configuration for all

Artur Hefczyc commented 9 years ago

I am not against the new config option but in my opinion it would server more as a tool for performance testing. A correct approach is to review what is the current default for each plugin and adjust the default based on the experience and test results from the task #3307.

The task shows that the default number of DB connections should be increased and then, number of threads for DB bound plugins should also be adjusted to match the number of DB connections. I think it makes sense to calculate number of DB connections based on a number of CPU cores, and then similar factor for the DB plugins.

Also, during the test number of threads for presences and pep were adjusted to the point when we got optimal performance.

I think the numbers we used should be put as defaults for plugins - of course defaults based on a number of CPUs.

wojciech.kapcia@tigase.net commented 9 years ago

I agree that we should think of better defaults for the DB pools (changed the subject) and plugins.

As for the configuration option - I think that in addition it could be a good idea as deployments tend to differ both in terms of hardware as well as configuration so coming up with "one to rule them all" may be difficult.

wojciech.kapcia@tigase.net commented 9 years ago

I've applied proposed changes:

  • revised plugins code, removed fixed int thread pools, increased default pools as per results from #3307;

  • created new options sess-man/sm-threads-factor[I]=1 to globally increase thread pool counts;

  • made database connection pool bound to the number of CPUs (factor 4 by default, no less than 10 connections, still over-ridable by current configuration options)

Artur Hefczyc commented 9 years ago

Good stuff, thank you.

issue 1 of 1
Type
New Feature
Priority
Normal
Assignee
RedmineID
3856
Estimation
8h
Spent time
7h 30m
Issue Votes (0)
Watchers (0)
Reference
tigase/_server/server-core#632
Please wait...
Page is in error, reload to recover