...
BugZero found this defect 3695 days ago.
In Veeam Backup & Replication, if more than 100 jobs are running simultaneously, and the user attempts to start more jobs, they may fail to start with the following error message: Error Job <JobName> cannot be started. SessionId: <UUID>, Timeout: [361 sec] A Veeam Agent for Microsoft Windows backup job may fail with: Managed session <UUID> has failed In the Veeam Agent for Microsoft Windows service logs (Svc.VeeamEndpointBackup.log) following errors are found: Error Job <JobName> cannot be started. SessionId: <UUID>, Timeout: [361 sec] Error Job [Infrastructure update] has failed to start. Timeout expired (6 minutes). SessionId: <UUID> The following event is found in the System event logs: Log Name: System Source: Win32k Event ID: 243 Task Category: None Level: Warning Keywords: Classic User: N/A Computer: Description: A desktop heap allocation failed.
By default, Veeam services are run as the LocalSystem account. As a result, all processes are started under NT AUTHORITY\SYSTEM context. If the Windows OS is running out of Desktop Heap memory, new processes can't be started.
The Desktop Heap size must be increased via a registry modification. Because this change will affect the desktop heap of all services (also known as non-interactive window stations), do not make the new value larger than necessary. Open Registry Editor (regedit) and navigate toHKLM\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems Within the SubSystems key will be a Value named Windows. Double-click on the value named Windows.
Starting with Server 2012 R2, Microsoft has changed the core logic of the heap memory management system, and publicly available tools used with previous versions to check per process head usage are incompatible. If you encounter the Veeam error listed in this KB and Event ID# 243 is found in the event viewer, we advise opening a case with Microsoft support to request assistance tracking down any service memory leaks.