Symptom
The IPSEC VPN tunnel flaps intermittently during a rekey request from Cisco ASA. The VPN recovers in a couple of minutes on its own, however, there is an outage for a couple of minutes when the tunnel is down.
Conditions
When there is an IPSEC tunnel formed between Cisco ASA and PAN firewall
Workaround
Increase IPSEC SA lifetime on ASA, so that the peer has lower time and ASA is rekey responder, not the initiator.
Further Problem Description
In IKEv2 debugs, we see that during rekey initiated by ASA, the first new inbound SPI is created, and then ASA sends information that the same SPI is invalid. This is not expected.
Jul 25 22:35:53 IKEv2-PLAT-5: RECV PKT [CREATE_CHILD_SA] [x.x.x.x]:500->[y.y.y.y]:500 InitSPI=0x48a0d49e589d838c RespSPI=0x9a973412d4dfc9df MID=00000008
Jul 25 22:35:53 IKEv2-PLAT-4: Received PFKEY Invalid SPI for SPI 0x8D5E4089, error FALSE
(229):
Jul 25 22:35:53 IKEv2-PROTO-4: (229): Received Packet [From x.x.x.x:500/To y.y.y.y:500/VRF i0:f0]
(229): Initiator SPI : 48A0D49E589D838C - Responder SPI : 9A973412D4DFC9DF Message id: 8
(229): IKEv2 CREATE_CHILD_SA Exchange RESPONSEJul 25 22:35:53 IKEv2-PROTO-5: (229): Next payload: ENCR, version: 2.0 (229): Exchange type: CREATE_CHILD_SA, flags: RESPONDER MSG-RESPONSE (229): Message id: 8, length: 224(229):
Payload contents:
Jul 25 22:35:53 IKEv2-PROTO-4: decrypt queued(229):
(229): Decrypted packet:(229): Data: 224 bytes
Jul 25 22:35:53 IKEv2-PROTO-7: Process delete IPSec API
Jul 25 22:35:53 IKEv2-PROTO-7: (229): SM Trace-> SA: I_SPI=48A0D49E589D838C R_SPI=9A973412D4DFC9DF (I) MsgID = 00000001 CurState: READY Event: EV_SEND_INVALID_SPI
Jul 25 22:35:53 IKEv2-PROTO-7: (229): Action: Action_Null
Jul 25 22:35:53 IKEv2-PROTO-7: (229): SM Trace-> SA: I_SPI=48A0D49E589D838C R_SPI=9A973412D4DFC9DF (I) MsgID = 00000001 CurState: INFO_I_BLD_INFO Event: EV_SEND_INVALID_SPI
Jul 25 22:35:53 IKEv2-PROTO-4: (229): Sending INVALID_SPI notify
Jul 25 22:35:53 IKEv2-PROTO-7: Construct Notify Payload: INVALID_SPIJul 25 22:35:53 IKEv2-PROTO-4: (229): Building packet for encryption.
(229):
Payload contents:
(229): NOTIFY(INVALID_SPI)(229): Next payload: NONE, reserved: 0x0, length: 12
(229): Security protocol id: ESP, spi size: 0, type: INVALID_SPI
(229):
(229): 89 40 5e 8d <---------------- new valid SPI
