General
After [non]pairwise key enabling and reboot, bfd ip and port mismatch on device leading to bfd down state
Symptom
SDWAN IPsec tunnels / SDWAN BFD could be in down state
Conditions
(case#1):
In race conditions TTM may send TLOC update for a existing tloc_index.
Ex: when remote TLOC is NATed (not SymNAT) and is rebooted; remote is under GR state
at the same time if local is also rebooted and came up faster than the remote;
vSmart will send the last known TLOC deatils if remote to local; which is old TLOC ( having old portNum)
Once remote also comes UP the vSmart sends latest deatils of remote TLOC.
if there was a change in portNum in remote TLOC; this results to update of the existing tloc_index with new portNum.
Problem Statement(case#2):
1. IPSec anti-reply drops at spoke due to excessive IPSec DELETE followed by CREATE on HUB router. Eventually it causes bfd session to go down.
Scenario to hit the issue:
2. The spokes are configured with private color tloc (like mpls or private1 etc).
3. The control connection of these private color needs to be behind NAT and the datapath does not have NAT.
4. On scale setups this race condition will be potentially exposed during rekey events
Workaround
Execute "clear omp all" on the BFD down devices
Further Problem Description
Remote TLOC is NATed; and has also gone through pop-hop hence has non-default port-number.
Both local and remote are rebooted at same time ; hence remote is under GR sate.
Local comes UP faster than remote and new BFD may fail to come UP.
