tl,dr;
There is an issue with registering/unregistering cluster nodes (wrong state, duplication)
Details:
After a flurry of monitor notifications it turned out that for some reasons some cluster nodes may not be correctly unregistered.
Current instances:
Instance Tigase group Name State Public DNS
i-070996b522632c5e2 tigase-main-xmpptigase-main-xmpp running ec2-52-35-187-124.us-west-2.compute.amazonaws.com
i-0ad9bc21f4b829342 tigase-main-xmpptigase-main-xmpp stopping ec2-18-237-84-154.us-west-2.compute.amazonaws.com
i-0af2254eaa6208fce tigase-main-xmpptigase-main-xmpp running ec2-34-220-178-249.us-west-2.compute.amazonaws.com
(stopping was the faulty one: Public DNS (IPv4): ec2-18-237-84-154.us-west-2.compute.amazonaws.com, Private DNS: ip-10-0-14-118.us-west-2.compute.internal)
tl,dr; There is an issue with registering/unregistering cluster nodes (wrong state, duplication)
Details:
After a flurry of monitor notifications it turned out that for some reasons some cluster nodes may not be correctly unregistered.
Current instances:
(stopping was the faulty one: Public DNS (IPv4): ec2-18-237-84-154.us-west-2.compute.amazonaws.com, Private DNS: ip-10-0-14-118.us-west-2.compute.internal)
Current cluster_nodes content:
And internal state says
while being connected to
ip-10-0-25-189.us-west-2.compute.internal
:And Tigase tries to deliver packets to stale connection: