...
A dataplane restart is observed which will result in failovers in A/S environments or disruption in A/A environmentsMonitoring the available memory in mbuf_pool_socket_0 shows a continuous fall in available memory edge01> get dataplane memory statsTue May 11 2021 UTC 15:48:09.928Memory UsageAvailable_entries : 51851 <<<< monitor available mbuf memoryAvailable_entries_in_cache : 204Cache_size_per_core : 128Name : mbuf_pool_socket_0Per_core_cache Available_entries : 75 Core_id : 0 Available_entries : 129 Core_id : 1Size : 77690 In /var/log/syslog, memory stats show the fall in available memory, monitor the second numeric field after Mempool 2021-04-16T11:39:29.120Z edge1 NSX 19492 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="stats" tname="stats22" level="INFO"] Mempool 1024 28564 24000 53236 16383 2061548 524288 524288 524288 524288 262144 18737 49152 8096 131072 1048576 4194304 1048576 1048576 1048576 4194304 343992021-04-16T11:49:29.120Z edge1 NSX 19492 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="stats" tname="stats22" level="INFO"] Mempool 1024 26168 24000 53236 16383 2063474 524288 524288 524288 524288 262144 18738 49152 8096 131072 1048576 4194304 1048576 1048576 1048576 4194304 343992021-04-16T11:59:29.120Z edge1 NSX 19492 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="stats" tname="stats22" level="INFO"] Mempool 1024 23675 24000 53245 16383 2063767 524288 524288 524288 524288 262144 18738 49152 8096 131072 1048576 4194304 1048576 1048576 1048576 4194304 343992021-04-16T12:09:29.121Z edge1 NSX 19492 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="stats" tname="stats22" level="INFO"] Mempool 1024 21293 24000 53241 16383 2063727 524288 524288 524288 524288 262144 18739 49152 8096 131072 1048576 4194304 1048576 1048576 1048576 4194304 343992021-04-16T12:19:29.121Z edge1 NSX 19492 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="stats" tname="stats22" level="INFO"] Mempool 1024 19195 24000 53246 16383 2065737 524288 524288 524288 524288 262144 18739 49152 8096 131072 1048576 4194304 1048576 1048576 1048576 4194304 34399 In /var/log/syslog logging indicates that datapathd has experienced an out of memory condition 2021-04-16T17:04:10.738Z edge1 NSX 28886 FABRIC [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="intel-rte" level="WARN"] KNI: Out of memory There is DHCP Request traffic in the environment which has a destination IP on the NSX-T Gateway however no DHCP service is configured on the Gateway.
A memory leak condition can occur on an Edge when invalid DHCP traffic targets an IP on the Edge but no DHCP services are configured on the Edge.
This issue is resolved in NSX-T Data Center 3.1.2, available at VMware Downloads .
To workaround this issue, the invalid configuration that results in these DHCP requests should be removed.Until the source of that traffic can be identified, a firewall rule can be used to immediately drop it.