Projects tigase _server server-core Issues #1472
Storage of Offline Messages to MongoDB (#1472)
Closed
Unknown opened 2 years ago

Isolating msg_history to mongoDB We are running our cluster on mysql, is it possible to "migrate" only the table msg_history from mysql to mongodb? Can we with config.tdsl change only the storage of offline messages from mysql to mongodb?

Details:

  • Tigase version: 8.2.1
Unknown commented 2 years ago

Hi, Yes, it's possible to configure different data-sources using different repositories and then configure actual repositories (in this case msgRepository) to use mongodb implementation and point it to use mongo://… data source.

There is no "migration" path available so you would have to manually move the messages from one database engine to the other.

Unknown commented 2 years ago

yes I know that I have to migrate old data manually, but the table is simple so I do not see any problem, thanks

wojciech.kapcia@tigase.net commented 3 weeks ago

New comments from

@davide.1 :

Hi, regarding this issue is tigase compatible also with https://aws.amazon.com/documentdb/ instead of MongoDB? Do you have already tested it? Or I need to check if all the operations used by tigase are supported by DocumentDb ?

@davide.1

Hi, here are the supported commands by DocumentDb: https://docs.aws.amazon.com/documentdb/latest/developerguide/mongo-apis.html

with low priority could you check if tigase is using only commands supported also by DocumentDb ? Thanks

wojciech.kapcia@tigase.net commented 3 weeks ago

Hmm... tigase-mongodb implementation is public and available here: https://tigase.dev/tigase/_server/tigase-mongodb/~files or https://github.com/tigase/tigase-mongodb/

I quickly checked the sources for the calls on MongoCollection and for example in MongoRepository.java I see calls like insertOne or updateOne which I don't see on DocumentDB list - I see plain insert or update but and I'm not sure if that includes insertOne or not (they have explicit ReplaceOne). Though looking at mongodb api it list insertOne() and insertMany() thus I would imagine that both would be supported in the end.

I quickly took a cursory look at the rest and I only found regular basic operations like insert/update/find.

wojciech.kapcia@tigase.net changed state to 'Open' 3 weeks ago
Previous Value Current Value
Closed
Open
Davide Marrone commented 3 weeks ago

ok, thank you for the links, before trying mongo I want to try fixing the stored procedure in the next few days, I think that maybe is enough to do that

Davide Marrone commented 7 days ago

In the last massive sending we haven't registered any problem also if there was some delay on AWS RDS for the massive insert/delete in the message archive table as ready seen in the other tickets. Also with 250 insert/delete per second tigase 8.1.2 was working correctly instead the 7.1.7 was not working correctly with this traffic, there was important delays up to 10 seconds, I guess that the problem was the wrong utilization of the database connection pool that was causing starvation for some connection. It seems that this problem has been fixed with the tigase 8.1.2 so I will not investigate further the migration to mondo/documentdb, you can close this ticket

wojciech.kapcia@tigase.net commented 4 days ago

Thank you Davide for the update.

wojciech.kapcia@tigase.net batch edited 4 days ago
Name Previous Value Current Value
State
Open
Closed
Assignee
empty
wojtek
issue 1 of 1
Type
Question
Assignee
Issue Votes (0)
Watchers (4)
Reference
tigase/_server/server-core#1472
Please wait...
Page is in error, reload to recover