They originally configured vSwitch0 for the 4 1 gb nic ports (these 4 are in a LACP Trunk Group on stacked Aruba 2390 switches) as vSwitch0 with "Teaming and Failover" set to "Route based on IP hash" and no traffic shaping. As far as we can tell we can only get 1gb of throughput on vSwitch0.
This is the primary reason why I (and most of the literature out there) advise people to not use LACP in a vSphere environment, because 1) they don't really understand how it works (1 Gb may be correct) and 2) there is better technology available.
Should this have worked?
No, this behavior is correct. If you're doing active LACP you have to set the teaming policy to route based on IP hash.
Thank you for the response.
Is there anyway i can set this up to be able to get 4GB throughput? - remove LACP?
maybe change the Load Balance algorithm on the Aruba to L4-Based?
Depending on your environment and the hosts involved in communication, you may never see 4 Gbps/sec (which is the theoretical maximum anyhow) of traffic. What I'd recommend is you spend some time reading how LACP works and understand how your hosts communicate on your network. LACP isn't a fire hose which is designed to evenly "spray" out traffic across all links. It just doesn't work that way.