Wojciech Kapcia wrote:

Folloup to #5682:

%kobit wojtek What do you think? Should we follow with rename of a config file? If so, let's create separate task for that.

As per request.

This is more RFC ticket.

I think that we should consider following:

• is there existing extension, which usage would justify the switch (for example better out-of-the-box syntax high-lighting)?

Indeed, it would be better to use some existing extension if possible. I am not that familiar with the DSL format to suggest something though. The syntax looks similar to many scripting languages. I guess if we used groovy extension some of the editors would already offer some syntax support.

• do we want to support multi-configuration files setup (like previously)? it was used to support different configurations for different databases previously (for example in TTS) but right now we have support for environment variables and there is tendency to rely more and more on the defaults (so, opposition to having more and more configuration files);

It was useful in some cases in the past. I am not sure if this is still relevant. I am personally in favor of not having a single, large configuration file which needs to be changed to update some settings.

It's easier to have kind of a "master" file which fairly static settings which can be relatively big and one or more small files with settings changed more often. On the other hand. How often a configuration is changed anyway?

Indeed, the most handy this is for the testing system, where we run tests over the system with almost identical settings and only small parts changed, such as DB or something like this.

Would it still be useful for our Test-NG system?

• developing on the above - do we need to configure the properties file name? I think majority of the installations use the default… we could hard-code the default (as right now - when the property or file from property is not present) and remove the hard-code from the etc/tigase.conf but still support overriding the default (if someone would like to point to init.properties in different location?).

Simpler is better in this case for me.

• I think that updating the documentation would be the least of the problems - old threads would be more problematic.

I still like your idea of not supporting any single case but just discovering that something is not right and notify a user, preferably with a link to correct documentation.

What I mean here. The new software could discover old configuration style or incompatible settings and does not need to handle it all transparently and automatically. It would be good enough to just print a warning with some explanation and pointers on where to find more information.

Artur Hefczyc wrote:

Wojciech Kapcia wrote:

I think that we should consider following:

• is there existing extension, which usage would justify the switch (for example better out-of-the-box syntax high-lighting)?

Indeed, it would be better to use some existing extension if possible. I am not that familiar with the DSL format to suggest something though. The syntax looks similar to many scripting languages. I guess if we used groovy extension some of the editors would already offer some syntax support.

%andrzej.wojcik any suggestion? From what I've read it's supposedly a groovy syntax, but there were a couple of changes (support for system properties and dashes in option names) hence it's not completely exact…

• do we want to support multi-configuration files setup (like previously)? it was used to support different configurations for different databases previously (for example in TTS) but right now we have support for environment variables and there is tendency to rely more and more on the defaults (so, opposition to having more and more configuration files);

It was useful in some cases in the past. I am not sure if this is still relevant. I am personally in favor of not having a single, large configuration file which needs to be changed to update some settings.

It's easier to have kind of a "master" file which fairly static settings which can be relatively big and one or more small files with settings changed more often. On the other hand. How often a configuration is changed anyway?

It's not, and I think majority of deployments use single config file (on, to be honest only TTS was using multiple config files…).

What's more, I think you've said that we should provide relatively best defaults that would limit the need for tweaks, hence better defaults -> smaller config file.

Indeed, the most handy this is for the testing system, where we run tests over the system with almost identical settings and only small parts changed, such as DB or something like this.

Would it still be useful for our Test-NG system?

I think not. With the recent changes it's better and simpler to user system properties for that (and now neither TTS nor TTS-NG use multiple configuration files - we simply use environment variable instead of a bunch of confing files just to store different JDBC property).

• developing on the above - do we need to configure the properties file name? I think majority of the installations use the default… we could hard-code the default (as right now - when the property or file from property is not present) and remove the hard-code from the etc/tigase.conf but still support overriding the default (if someone would like to point to init.properties in different location?).

Simpler is better in this case for me.

• I think that updating the documentation would be the least of the problems - old threads would be more problematic.

I still like your idea of not supporting any single case but just discovering that something is not right and notify a user, preferably with a link to correct documentation.

What I mean here. The new software could discover old configuration style or incompatible settings and does not need to handle it all transparently and automatically. It would be good enough to just print a warning with some explanation and pointers on where to find more information.

There is already an error message that should be more clear (vide #5005#note-9), i.e. indicating that particular configuration was not correct.

To summarize:

• we could remove configuration file from tigase.conf and hardcode;

• we don't need to support multiple configuration file;

• we should choose more appropriate extension (that would offer better syntax highlighting in most editors).

Anything missing?

As %wojtek mentioned, our DSL syntax is similar to groovy but it is not the same, so using groovy extension may lead to the errors being reported, while in fact configuration would be correct.

Nothing more from my side.

What about '.dsl' extension then?

I might be able to create a syntax processor/highlighter for emacs if I have time. Not sure who else, except me would use it though.

I think some of you use vim. You can ':set ft=groovy' or java, or anything else to change syntax highlighting.

Artur Hefczyc wrote:

What about '.dsl' extension then?

It seems this one exists already so maybe .tdsl (tigase dsl)?

If noone (%kobit %bmalkow %Eric %Daniel ) opposes then %andrzej.wojcik, could you:

• remove configuration file from tigase.conf (but leave option to specify it there) and hardcode file name;

• update converter: use configured .properites file from older version, convert it and store in .tsdl file, rename .properties file

Looks OK to me, please assign ticket to me once it's finished so i know when to update documentation.

%kobit What do you think about renaming config file?

As we already decided to change suffix from .properties to .tdsl it would be nice to rename the file as well. Now it is named init.properties which does not reflect its content or purpose. It is a configuration file, so maybe we could rename it to config.tdsl or @server.tdsl@?

config.tdsl

I've changed the name of a config file, adjusted converter (with refactoring to move properties conversion to separate class) and modified web setup to use the proper file. This way it should be easier in the future to drop code used for conversion from properties file to DSL.

Additionally, I've introduced new config option --config-file which should be used to pass location and name of a DSL file. If not passed then etc/config.tdsl will be used.

Old config option --property-file is only used during conversion and after it is not used anymore.

Config file can be updated and renamed if needed when:

• scripts/tigase.sh upgrade-config etc/tigase.conf is called

• Tigase XMPP Server is started

There is a weird issue ocuring now - after stopping the server the file is reformatted and all non-normative settings are stripped (which includes commented out options). This is not ok. It's ok to run a one-time conversion but afterwards we should not touch/update configuration file.

I tried to replicate this issue and was unable to do so.

Right now we have 2 conversions which are taking place:

• conversion from properties file

Done if init.properties file exists. It overrides config.tdsl and init.properties is renamed to init.properties.old.X where X is next available backup number.

• upgrade of DSL configuration

DSL is loaded and checked if any field/property needs to be converted (ie. due to rename of a property).

My guess is that in your case the issue is with second conversion but in my case it is not executed.

Please attach your config file which is being "replaced" every time. I need it to find cause of this issue - ie. find why converted is assuming that there was an upgrade if there wasn't one.

I've just found an example config file which causes this issue during the upgrade of one of our servers. I need to check why this issue happens.

Moreover, during this upgrade, it was detected that SchemaManager was not adjusted to the usage of the new configuration file (still tries to load init.properties file).

Fixed issue with constant upgrades and issue with Schema Manager.

Still happening.

I'm on the latest sources:

$ git log --oneline --max-count=1 | cat
adbeb36c #5800: removed deprecated classes scheduled for removal in 7.2.0

I've added explicit config file configuration (to avoid annoying changing default repository config and avoiding accidental commit):

--config-file etc/my_config.tdsl

Server starts:

2017-07-17 13:25:31.665 [main]             ConfiguratorAbstract.parseArgs()        CONFIG:   Setting defaults: --config-file = etc/my_config.tdsl
2017-07-17 13:25:31.702 [main]             OldConfigHolder.detectPathAndFormat()   FINEST:   No property file not specified! Using default one etc/init.properties
2017-07-17 13:25:31.702 [main]             OldConfigHolder.detectPathAndFormat()   FINEST:   Provided property file /Users/wojtek/dev/tigase/tigase-server/server/etc/init.properties does NOT EXISTS! Trying to use default one etc/init.properties
2017-07-17 13:25:31.788 [main]             ConfigHolder.loadFromDSLFiles()         CONFIG:   Loading configuration from file: etc/my_config.tdsl
2017-07-17 13:25:31.815 [main]             ConfigHolder.loadConfiguration()        CONFIG:   Configuration file etc/my_config.tdsl was updated to match current format. Previous version of configuration file was saved as etc/my_config.tdsl.old.3
2017-07-17 13:25:31.818 [main]             ConfigHolder.loadConfiguration()        CONFIG:   Loaded configuration:

I reviewed config file you've sent me and it required single time conversion which was done and after that no further conversions were done.

~~Seems to be working ok now. I've notice one more property that's not updated: @--script-dir@~~ (wrong ticket)