Type |
Task
|
Priority |
Normal
|
Assignee | |
RedmineID |
1694
|
Spent time |
4h
|
Issue Votes (0)
Watchers (0)
Type |
Task
|
Priority |
Normal
|
Assignee | |
RedmineID |
1694
|
Spent time |
4h
|
The basic question here is how to properly handle a notification to let's say 10M or 100M subscribers? Or generally how to handle notifications to a high traffic PubSub installation without a OOM error? Let's say we have 100M PubSub nodes with 100 subscribers each but publishing to nodes is quite frequent.
The main goal is to avoid OOM. It is acceptable to drop some packets in order to prevent queue overload as it may happen the generated traffic exceeds physical capabilities of the HW the PubSub is running on.
I was experimenting with a code which monitors memory usage and sleeps until it is low enough before generating more packets. Something similar could be implemented in PubSub I guess: Look at createNodes method