Symptom

When interface is down, asa HA model switches over, BFD multihop session is for eBGP and iBGP peers, bfd is down before it times out, which causes bgp down. Topology DUT1->FW1 ->DUT2 and DUT1->DUT3->FW2->DUT4->DUT2 1/ In the network environment, DUT1 and DUT2 are neighbors for both eBGP and multihop BFD . 2/ There are 2 paths for DUT1 and DUT2 to become neighbor. One is DUT1 ->FW1 -> DUT2, the other one is DUT1 -> DUT3 -> FW2 -> DUT4 ->DUT2. 3/ FW1 and FW2 are HA module. FW1 is active, FW2 is standby. Normally, the neighbor relationship btw DUT1 and DUT2 are built by path DUT1 ->FW1 -> DUT2, only when this path is broken, FW2 will become active, and the neighbor will be built by path DUT1->DUT3->FW2->DUT4->DUT2. 4/ When customer tried to shutdown/ no shutdown the connection btw DUT1 and DUT2 (DUT1 ->FW1 -> DUT2) for multiple times, BFD will be down for one time, which result in BGP being down. 018 Sep 11 10:37:43 DUT1 %SYSLOG-1-SYSTEM_MSG : Logging logfile (messages) cleared by user 2018 Sep 11 10:38:06 DUT1 %ETHPORT-5-IF_DOWN_CFG_CHANGE: Interface Ethernet3/33 is down(Config change)<<<<<<<<<<<<<<<<<< 2018 Sep 11 10:38:07 DUT1 %ETHPORT-5-IF_DOWN_ADMIN_DOWN: Interface Ethernet3/33 is down (Administratively down) 2018 Sep 11 10:40:28 DUT1 %ETHPORT-5-IF_ADMIN_UP: Interface Ethernet3/33 is admin up . 2018 Sep 11 10:40:32 DUT1 %ETHPORT-5-SPEED: Interface Ethernet3/33, operational speed changed to 1 Gbps 2018 Sep 11 10:40:32 DUT1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet3/33, operational duplex mode changed to Full 2018 Sep 11 10:40:32 DUT1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet3/33, operational Receive Flow Control state changed to off 2018 Sep 11 10:40:32 DUT1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet3/33, operational Transmit Flow Control state changed to off 2018 Sep 11 10:40:32 DUT1 %ETHPORT-5-IF_UP: Interface Ethernet3/33 is up in mode access 2018 Sep 11 10:42:02 DUT1 %ETHPORT-5-IF_DOWN_CFG_CHANGE: Interface Ethernet3/33 is down(Config change) 2018 Sep 11 10:42:03 DUT1 %ETHPORT-5-IF_DOWN_ADMIN_DOWN: Interface Ethernet3/33 is down (Administratively down) 2018 Sep 11 10:42:32 DUT1 %ETHPORT-5-IF_ADMIN_UP: Interface Ethernet3/33 is admin up . 2018 Sep 11 10:42:36 DUT1 %ETHPORT-5-SPEED: Interface Ethernet3/33, operational speed changed to 1 Gbps 2018 Sep 11 10:42:36 DUT1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet3/33, operational duplex mode changed to Full 2018 Sep 11 10:42:36 DUT1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet3/33, operational Receive Flow Control state changed to off 2018 Sep 11 10:42:36 DUT1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet3/33, operational Transmit Flow Control state changed to off 2018 Sep 11 10:42:36 DUT1 %ETHPORT-5-IF_UP: Interface Ethernet3/33 is up in mode access 2018 Sep 11 10:42:50 DUT1 %BGP-5-ADJCHANGE: bgp-100 [20125] (default) neighbor 1.1.200.1 Down - bfd session down <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<bfd down 2018 Sep 11 10:42:50 DUT1 %BFD-5-SESSION_STATE_DOWN: BFD session 1090519175 to neighbor 1.1.200.1 on interface has gone down. Reason: Control Det ection Time Expired in vrf default.

Conditions

BFD multihop session is for eBGP and iBGP peers, when one of the link is down, bfd is down before it times out, which causes bgp down. When eth 3/33 is shut on DUT1, This event notification comes to BFD from L2fm module , which inturn call adjacency manger API This API gives us 3/22 as new interface to be used for Vlan 100 for BFD packets to TX. As per DUT1 all is well. On DUT2, Also BFD gets notification from L2fm module for this change, however the adjacency manager API is still giving us eth 3/33 as outgoing Interface, due to which TX packets on DUT2 still sent on 3/33 which are not reaching DUT1 thus MHBFD flaps. 64) Event:E_DEBUG, length:38, at 104086 usecs after Tue Sep 11 20:18:30 2018 [108] [1040]:SVI/Access port vlan 200 65) Event:E_DEBUG, length:25, at 103967 usecs after Tue Sep 11 20:18:30 2018 [108] [953]:Intf: Eth3/33 <<ß------------? am_resolve_l3_addr? gives us eth3/33 66) Event:E_DEBUG, length:70, at 103959 usecs after Tue Sep 11 20:18:30 2018 [108] [929]:AM retval:0x0 for iod:92 l3_afi:1 addr:17010102 iod_out:70 74) Event:E_DEBUG, length:58, at 103864 usecs after Tue Sep 11 20:18:30 2018 [108] [447]:New: Intf Eth3/22 : 0x1a115000 Mac 01c.54ff.82 <<<ß MTS_OPC_L2FM_MAC_MOVE_NTFN 75) Event:E_DEBUG, length:37, at 103860 usecs after Tue Sep 11 20:18:30 2018 [108] [436]:Intf Vlan200 : 0x90100c8 76) Event:E_DEBUG, length:62, at 103835 usecs after Tue Sep 11 20:18:30 2018 [108] [411]:Got a MAC notification from L2FM for 1 adjacencies When interface is down, L2FM and AM are both notified with the info, which causes the bfd session is interrupted before it timed out, Normally, the information that interface being down should only notify L2FM.

Workaround

NA

Further Problem Description