Category: Cisco ACI General

So you may wonder why you should consider purchasing a standby APIC controller for your current APIC cluster. A valid argument is why you would need a standby APIC if you already have a Cisco 24x7x4 Smartnet contract. Well Smartnet is great but:

1. We are currently experiencing hardware shortages so sometimes Cisco cannot commit to getting you a replacement APIC in 4 hours or less.
2. If you do get an APIC replacement, it will most likely not ship with the code version of your current APIC cluster. As a result, you will spend time updating the CIMC ~1 hour + updating the APIC application code version to match. The time it takes depends on the code versions. In some cases, the appliance may ship with a newer code version and you will have to downgrade as well. You cannot assume that Smartnet 24x7x4 will mean you will have an operational APIC with in 4 hours because code upgrades/downgrades will be required.
3. When you get a replacement APIC, you must still take time to UNRACK the failed APIC and Install the new APIC so there is some time with physical labor to provision the replacement.
4. If you had a standby APIC, it will get automatically upgraded during your upgrade cycle so if you have a failed APIC in the cluster, you can easily replace the failed with the Standby APIC controller. No new physical connections are needed as all of this has already been planned and provisioned.

Recently I came across a request to rename the APIC Controllers and directly after the APICs were renamed and rebooted, OOB management access started flaking out. Basic PING tests revealed OOB reachability issues. What I discovered was that the Bond1 interface was consistently failing over between Eth1/1 and Eth1/2, the OOB management would continuously relearn the MAC addresses on these ports and this created a management access issue.

The ACI code version this was experienced with was 5.2(6e)

I confirmed this by checking where the BOND1 MAC address was being learned and after multiple refreshes on the management switch, it was obvious that it was flapping back and forth. This caused the PING tests to fail from time to time.

I did some web surfing for this issue and found some reference from an older 1.x version around LLDP issues when changing the hostname and the Cisco fix was to shutdown one of the BOND1 interfaces. But this is not acceptable for my customers, so I started doing some digging and the ultimate fix was to decommission each APIC at a time and wipe it. Then re-add the APICs to the cluster and after these steps were performed, the APIC OOB MAC Flapping was resolved.

STEPs

1. Document the preferred hostname, Fabric name, Fabric ID, POD #, VTEP pool, OOB MGMT IP and Gateway, the Infra VLAN used and the local admin credentials. These are the parameters you will need to recommission the APICs.
2. Decommission one APIC at a time from the cluster. Wait ~5 minutes to make sure its all replicated properly.
3. Console via CIMC into the target APIC and Wipe it out using below:

apic# acidiag touch clean
This command will wipe out this device. Proceed? [y/N] y
apic# acidiag touch setup
This command will reset the device configuration, Proceed? [y/N] y

apic# acidiag reboot
This command will restart the this device, Proceed? [y/N] y

3. After the APIC reboots, the SETUP should run and you can reprovision the APIC via CIMC using STEP 1’s parameters. Wait ~2-3 minutes for the APIC to converge itself after the settings.

4. Finally, commission the target APIC in the GUI by simply right-clicking the old APIC in the list and clicking commission. Even though the old APIC name may be shown, after ~5 minutes, the recommissioned APIC will converge into the cluster and be shown properly.

I am also asked sometimes how the OOB management switches should be configured for the APIC OOB BOND1 interfaces, since they are “bonded”. These ports should be treated similar to regular access ports.

!---OOB-MGMT-SWITCH---- ! 
interface G1/0/1  
 desc <APIC-HOSTNAME> OOB Bond1 management interface  
 switchport mode access  
 switchport access vlan <OOB-MGMT-VLAN>  
 spanning-tree portfast 
!

---

On March 6th, I will be hosting an Advanced Cisco ACI event where I will cover the benefits of Cisco ACI, go over real-world Cisco ACI Fabric and Tenant naming conventions, review brownfield migration strategies and demo how easy it is to inject firewalls to inspect traffic between migrated networks. In addition, we will also have our Cisco Learning Partner that will provide a Cisco ACI training.

---

https://lp.cdillc.com/2020-3-6-hcaf-avengers-aci-registration.html

---

With most Cisco routing platforms, by default, routers do not advertise network prefixes to eBGP peers whose AS is already found last in the BGP’s network prefix AS_PATH attribute. This is a loop prevention mechanism known as disable-peer-as-check.

Recently I deployed a number of Fortigate Fortinet firewalls within Cisco ACI and noticed that Fortigates (with code 5.6) do not have any option of enabling this feature. I was actually surprised because even Palo Alto firewalls have this as an option. In Palo Alto, this option is known as “Enable Sender Side Loop Detection” and its found directly under the neighbor configuration. When you deploy a lot of VRFs with BGP in ACI, this is an option you need to be aware of.

Recently I deployed a Cisco ACI 4.2 Multi-Pod deployment, or let’s say I added a secondary POD to a Cisco ACI 4.2 deployment. One of the things I noticed is that you now have to click on Add a Pod under Fabric Inventory and its all wizard based which is a pain, especially if you already have GOLF configured and you want to use the same MPOD L3OUT.

If you want to use the same interface Multi-pod is using and leverage GOLF on the same interface, you cannot use the wizard. You have to manually specify the POD2 spine interfaces and pod info under the GOLF L3out as well as the Fabric access policies and assign the interface profiles to the spine switch profiles.

Why is this secondary POD Spine information important? Well because if you do not enter the secondary POD Spine information into the L3OUT and Fabric External Connection Policies, when you register the Spine under Fabric membership, the switch will be stuck in the Discovering stage and will will never be assigned a Infra TEP IP. It will look like its working but the spine will be stuck in Discovery. If you troubleshoot, you will notice discoveryissues on the switch, APIC DHCP does not issue DHCP IPs and it is stuck discovering.

Hopefully Cisco makes adding a POD with existing GOLF configuration easier to manipulate.