I added docker compose documentation to our main distribution repository (tigase-server now) under dedicated docker directory with docker-compose.yaml file: https://tigase.dev/tigase/_server/tigase-server/~files/master/src/main/docker/README.md (linked from main page) and updated our docker hub page: https://hub.docker.com/r/tigase/tigase-xmpp-server

I also updated the TURN/STUN guilde: https://tigase.org/blog/tigase-server-with-stun-turn/

(While updating the guide I had to add 1dev build job so the site would be automatically updated and fixed a couple of issues with permissions)

This allowed me to create and run docker compose environment with def settings. Except I used "nightly" tag and custom hostname.

However, in the configuration wizard I was not able to setup the system. I got stuck on DB. I selected MySQL and provided host, root user, pass from the yaml file (would be nice for the configurator to automatically pickup these settings from env if possible). But clicking next, leads to an empty page with: java.lang.reflect.InvocationTargetException

I also tested latest and it all worked as expected. It looks like the problem is with current nightly.

I think the ticket can be closed now.

Works like a dream. Looking forward to test the 8.4 with the new dashboard.

This allowed me to create and run docker compose environment with def settings. Except I used "nightly" tag and custom hostname.

However, in the configuration wizard I was not able to setup the system. I got stuck on DB. I selected MySQL and provided host, root user, pass from the yaml file (would be nice for the configurator to automatically pickup these settings from env if possible).

It's planned: https://tigase.dev/tigase/_server/tigase-http-api/~issues/142, I think I should be able to add it soon as part of 8.4 release.

The idea is to have "next-next-next" setup experience without having to change too much and get the system up and running.

But clicking next, leads to an empty page with: java.lang.reflect.InvocationTargetException

Did you happen to get the whole stacktrace (it should be in the container logs)?

I do not have any stackrace but I could recreate the problem if necessary.

Let me know when the version 8.4 is working, I would like to see and test the new dashboard.

I do not have any stackrace but I could recreate the problem if necessary.

No need - while working on the env I got the stacktrace.

@kobit There were a couple of issues to fix but I think I squashed all of them so now getting latest docker-compose.yaml, adjusting if needed (hostname) and then docker compose up should give you "next-next-next" working web-installer experience.

Did you mean latest or nightly? Is this code already committed? I tried latest today but it did not work as expected. If you confirm which tag I should try, I can provide all the details.

Also, is the installer available on localhost only or can be also accessed by a real IP address?

Did you mean latest or nightly? Is this code already committed? I tried latest today but it did not work as expected. If you confirm which tag I should try, I can provide all the details.

nightly as we still not released 8.4.0

latest in general refers to latest stable release

Also, is the installer available on localhost only or can be also accessed by a real IP address?

In general Tigase http server binds to all ports so you can access it via any hostname/IP that points to it

Btw. @kobit I was pondering - given leaning more towards using docker and docker compose maybe we could drop (in version 9.0) Derby support? I think noone is using it and it gives little to no benefit for us (we already use docker for any testing so it's very easy to get up and running without any conflicts). Maybe we could switch default in docker-compose to postgres, which should be lighter than mysql even?

Yes, I agree, good idea.

I meant dropping Derby support.

I have just tried nightly and it still does not work as expected. My guess is that nightly is still the old version without your recent changes.

I meant dropping Derby support.

Created https://tigase.dev/tigase/_server/server-core/~issues/1526

I have just tried nightly and it still does not work as expected. My guess is that nightly is still the old version without your recent changes.

Have you run docker compose pull? Could you share output of docker compose images? Could you also share initial log from the server for general server component info:

xmpp-cl1-1  | componentInfo{Title=Tigase XML Tools, Version=4.3.0-SNAPSHOT-b321/895a364b, Class=tigase.xml.XMLUtils}
xmpp-cl1-1  | componentInfo{Title=Tigase Utils, Version=4.4.0-SNAPSHOT-b480/35bbed6a, Class=tigase.util.ClassUtil}
xmpp-cl1-1  | componentInfo{Title=Tigase XMPP Server, Version=8.4.0-SNAPSHOT-b6354/81eb598b, Class=tigase.server.XMPPServer}
xmpp-cl1-1  | componentInfo{Title=Tigase XMPP Server Distribution, Version=8.4.0-SNAPSHOT-b12554/c2514465, Class=tigase.dist.XmppServerDist}

as well as http-api component version:

xmpp-cl1-1  | [2024-05-16 13:59:02:737] [INFO    ] [                  main ] BasicComponent.initialize()      : Loading component: http :: componentInfo{Title=Tigase HTTP API component, Version=2.4.0-SNAPSHOT-b461/836903cc-8.4.0-SNAPSHOT-b6354/81eb598b, Class=tigase.http.HttpMessageReceiver}

I removed all the previous images and now it works like a dream!!!!!. Dashboard works and looks very good too.

The only, small, concern is with the DB schema creation. It takes some time, like a minute or so. And after clicking next nothing happens during this time. Some ppl can be confused and impatient. I wonder if there is a way we could display some info or warning that the DB schema creation is in progress and it may take some time.

I removed all the previous images and now it works like a dream!!!!!.

Great.

Btw. I was thinking whether we shouldn't use postgres by default (especially in the "tiny fast deployment" kinda scenario) as it should use less resources.... (~500M vs ~85M)

mysql:

CONTAINER ID   NAME         CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O     PIDS
5348c9c4aa82   tmp-xmpp-1   8.53%     850.7MiB / 7.755GiB   10.71%    8.96MB / 33.1MB   0B / 172kB    1164
f75faa431395   tmp-db-1     0.92%     486.8MiB / 7.755GiB   6.13%     33.1MB / 8.96MB   0B / 12.3kB   77

postgres:

$ docker compose stats --no-stream
CONTAINER ID   NAME         CPU %     MEM USAGE / LIMIT     MEM %     NET I/O         BLOCK I/O     PIDS
3e179e920dbb   tmp-xmpp-1   0.53%     1.105GiB / 7.755GiB   14.25%    126kB / 139kB   0B / 65.5kB   1198
ed66241c2db9   tmp-db-1     0.08%     84.32MiB / 7.755GiB   1.06%     140kB / 124kB   0B / 414kB    47

The only, small, concern is with the DB schema creation. It takes some time, like a minute or so. And after clicking next nothing happens during this time. Some ppl can be confused and impatient. I wonder if there is a way we could display some info or warning that the DB schema creation is in progress and it may take some time.

Weird... It takes like 2-3s on my machines.

xmpp-1  | [2024-05-20 13:35:41:383] [CONFIG  ] [    http-server-pool-1 ] DBSchemaLoader.init()            : Parameters: [adminPassword: ****, admins: null, dbHostname: db, dbName: tigasedb, dbPass: ***********, dbRootPass: ****, dbRootUser: null, dbRootAsk: false, dbType: postgresql, dbUser: tigase_user, file: null, ingoreMissingFiles: null, logLevel: CONFIG, query: null, serverTimezone: null, useLegacyDatetimeCode: null, useSSL: null, forceReloadSchema: false, schemaDirectory: database/, otherParameters: {}]
xmpp-1  | [2024-05-20 13:35:41:891] [CONFIG  ] [    http-server-pool-1 ] DBSchemaLoader.init()            : Parameters: [adminPassword: *****, admins: [admin@atlantiscity], dbHostname: db, dbName: tigasedb, dbPass: ***********, dbRootPass: ****, dbRootUser: root, dbRootAsk: false, dbType: postgresql, dbUser: tigase_user, file: null, ingoreMissingFiles: false, logLevel: CONFIG, query: null, serverTimezone: null, useLegacyDatetimeCode: true, useSSL: null, forceReloadSchema: false, schemaDirectory: database/, otherParameters: {}]
xmpp-1  | [2024-05-20 13:35:41:892] [INFO    ] [    http-server-pool-1 ] DBSchemaLoader.validateDBConnection(): Validating DBConnection, URI: jdbc:postgresql://db/postgres?user=root&password=****
xmpp-1  | [2024-05-20 13:35:42:045] [INFO    ] [    http-server-pool-1 ] DBSchemaLoader.validateDBConnection(): Connection OK
…
xmpp-1  | [2024-05-20 13:35:42:758] [INFO    ] [    http-server-pool-1 ] DBSchemaLoader.lambda$postInstallation$12(): Finalizing...
xmpp-1  | [2024-05-20 13:35:42:762] [INFO    ] [    http-server-pool-1 ] DBSchemaLoader.lambda$postInstallation$12():  completed OK
xmpp-1  | [2024-05-20 13:35:42:763] [INFO    ] [    http-server-pool-1 ] ConnectionLock.unlock()          : Unlocking database
xmpp-1  | [2024-05-20 13:35:42:763] [INFO    ] [    http-server-pool-1 ] ConnectionLock.cleanup()         : Closing lock connection

Do you use native images or maybe wrong platform and emulation is used? How long it takes for Tigase to start?

xmpp-1  | [2024-05-20 13:29:17:460] [INFO    ] [                  main ] XMPPServer.start()               : Server finished starting up in (12s) and (if there wasn't any error) is ready to use

We could add a notice/info/warning on DB configuration page that installation may take a while and page should not be refreshed/reloaded - that should be quite simple and easy to add (as opposed to page with progress)

The only, small, concern is with the DB schema creation. It takes some time, like a minute or so. And after clicking next nothing happens during this time. Some ppl can be confused and impatient. I wonder if there is a way we could display some info or warning that the DB schema creation is in progress and it may take some time.

We could add a notice/info/warning on DB configuration page that installation may take a while and page should not be refreshed/reloaded - that should be quite simple and easy to add (as opposed to page with progress)

@kobit I added notice/info to the web-installer

Sure, in such a case I am in favor of using postgresql instead of mysql. This is kind of strange, I was always under impression that mysql is the DB with little resources requirements. 0.5GB RAM for such a small setup seems really excessive.

I am running the setup on my NAS. But this is really powerful setup - Intel Core i7-7700 3.6 GHz, 64GB RAM. I do not have any performance issues with anything on it. Maybe loading schema to postgresql would be faster? Maybe there is some issue with resources limits which slows down mysql? In any case, after loading schema all works fine, so it does not concern me much.

Just tested the latest nightly. The added working is good, and enough. This time schema loaded within a few seconds.

To sum it up. I consider the task finished and fully working, you can close the issue if there is nothing else to do.

As for the PostgreSQL, I understand that we can simply have docker compose for MySQL and another one for PostgreSQL?

Sure, in such a case I am in favor of using postgresql instead of mysql. This is kind of strange, I was always under impression that mysql is the DB with little resources requirements. 0.5GB RAM for such a small setup seems really excessive.

Maybe in the past it required less resources but AFAIR for a while now PostgreSQL is considered as more efficient... However I did quick digging and from https://www.startechup.com/blog/postgresql-vs-mysql/ I get:

PostgreSQL creates a new system process for every client connection, using a lot of memory for multiple connections. In contrast, MySQL uses a single process and a single thread per connection, making it better for smaller applications. thus the conclussion is "It depends". Probably MySQL could be tweaked to use less memory (less connections from Tigase, lower buffers, possibly disable "performance schema" but - PostgreSQL seems better "out of the box" here and probably in lot's of other cases thus I still think it would be better default.

As for the PostgreSQL, I understand that we can simply have docker compose for MySQL and another one for PostgreSQL?

Yes. We can have both: just distinct files like docker-compose.mysql.yaml and docker-compose.postgresql.yaml - in the end those are just plain text files with specification what and how to deploy. I was pondering doing profiles (like with coturn) but I wasn't sure how to handle different environment variables - most problematic being db type and port (afair it's not possible to specify the application port so only service port mapping is available).

I think the most sensible option would be having two compose files: docker-compose.yaml (default, postgres) and docker-compose.mysql.yaml (as a flavour)

Ok, we have a working docker compose and it works very well. And, additional versions or options are not a priority right now. I leave it to you to decide whether you want to spend some more time on this or close the issue.

TBH, for the use-case like mine I do not need many DB connections, so probably postgresql with just few DB connections would work well enough, saving memory. But, the current MySQL option works well enough.

In the end I switched the default to postgresql and created alternative .mysql one. I think this concludes the task.