So I have a three node Log Insight cluster and enabled the Integrated Load Balancer, entered the IP and FQDN which my clients are pointing to. All good so far.
I'd like to understand how the load balancing actually works, i.e. If one of the nodes becomes unavailable id still expect to be able to ping the ILB address? the behavouir I'm seeing at the moment is when the Master is down so is the ILB address, is this expected?
Yes if your ILB IP was tied to the master when it went down, the ILB IP is taken over by another node which is why ingestion continues. Which means you can ping the IP, but if you access the UI via the VIP it tries to go to the master's UI but since master is dead the UI is not available.
When you have a 3 node cluster (or any size for that matter), You should be able to login to the IP of another node and check which node is holding the IP.
Having said that , when you have a 3 node cluster and 1 node goes down, that makes Cassandra (database) very unhappy. ( Note 2 node clusters are not supported).
Does this node that is down come back up after a while? or does down imply it is dead for good?
Some details about the Log Insight ILB and how it works is here -
Log Insight: Testing Integrated Load Balancer Failover - SFlanders.net
12 Reasons Why You Should Use The Log Insight Integrated Load Balancer - SFlanders.net
Thanks for that just wanted to clarifiy something:
The ILB role is being transferred if one of the nodes fails but the Master role does not get transferred, does this sound right?
Correct Master does not get transferred. With Log Insight 3.0 and newer Log Insight has moved to a distributed architecture model instead of a master/slave model but a few things still remain master/slave. If the Master node is down then while the UI remains available, ingestion continues but the Cluster page does not remain available and upgrades are not poossible.
Ok that makes sense with what I'm seeing in my testing with the three node 3.0 cluster.
It would also appear that with the Master node unavailable I am unable to get to the Cluster LIB login page , i.e loginsight.test.local. However I can still ping the ILB IP address, when i logon to one of the two remaing nodes I can see events are still being ingested between the two remaining appliances. Is the unavailbility of the LIB web address expected?
Yes if your ILB IP was tied to the master when it went down, the ILB IP is taken over by another node which is why ingestion continues. Which means you can ping the IP, but if you access the UI via the VIP it tries to go to the master's UI but since master is dead the UI is not available.
Ok thanks, is the behaviour the same in 3.3.1?
Yes it is.