- After migrating Windows 2012 bare-metal servers configured with “Switch Independent NIC teaming” with “dynamic” distribution, Cisco ACI experienced rapid faults which would continue to be Raised and Cleared continuously.
- Faults for various hosts
- “ACI has detected multiple MACs using the same IP Address x.x.x.x”
- Reason for Fault:
- Windows NIC Teaming using Switch Independent Mode with dynamic load-balancing mode distributes network traffic load based on the TCP Ports address hash as modified by the Dynamic load balancing algorithm. The Dynamic load balancing algorithm redistributes flows to optimize team member bandwidth utilization so that individual flow transmissions can move from one active team member to another.
- This results in ACI detecting multiple MACs using the same IP address on all Baremetal server uplinks.
- In addition, other COOP faults are seen as ACI freezes Endpoints which cause a disruption and the bare-metal server becomes unavailable.
- Faults for various hosts
- FIX:
- 2-3 Options
- Easy Fix
- Modify the load balancing algorithm from dynamic to HyperVPort on the Windows bare-metal server.
- Get-NetLbfoTeam | Set-NetLbfoTeam -LoadBalancingAlgorithm HyperVPort
- Don’t worry, you don’t need HyperV deployed. With this algorithm, all traffic entering and exiting any given adapter will always use the same physical adapter.
- Modify the load balancing algorithm from dynamic to HyperVPort on the Windows bare-metal server.
- Preferred Fix
- Configure Switch Dependent NIC Teaming using either Static Etherchannel or LACP.
- This is more time consuming on the ACI side since each bare-metal servers requires a unique port-channel policy group and this policy group needs to be separately bound to the respective EPG.
- Configure Switch Dependent NIC Teaming using either Static Etherchannel or LACP.
- Temporary Band-Aid
- Disable the secondary uplinks on each host so only 1 uplink is used per host until a solution is solidified.
- Easy Fix
- 2-3 Options
Practice Not Theory