Same EVC level, still no VMotion

To save on Microsoft SQL licenses, we decided to merge two VMware Clusters into one. The clusters were in different vCenters and the source Cluster was running ESXi 6.5u2 and the target cluster was ESXi 6.5u1. Both clusters were at the same Westmere EVC Level but a Cross vCenter VMotion failed with “The target host does not support the virtual machine’s current hardware requirements.”. . . To resolve CPU incompatibilities, use a cluster with Enhanced vMotion Compatibility (EVC) enabled. See KB article 1003212.”.

Reading the release notes on 6.5u2 and with some input from this blogpost: “VMotion fails to migrate VMs between ESXi hosts which have the same configuration” I concluded that I should bring the target host to the same 6.5u2 build as the source host. So I upgrade on host in the target cluster to 6.5u2.

Testing in a new cluster

Unfortunately, the Cross vCenter VMotion still failed. Together with VMware Support we discovered that as long as there is one 6.5u1 host in the cluster, the Westmere EVC level masks the “cpuid.SSBD” feature. I performed the following test: Create new cluster, put in a 6.5u2 host and set the EVC to Westmere. Setup a SSH session into the host and run the following command:

vim-cmd hostsvc/hostconfig | grep -E -A1 “SSBD|IBPB|STIBP|IBRS”

Snippet of the output:

     key = "cpuid.SSBD",
     featureName = "cpuid.SSBD",
     value = "1"

You can now see that the CPUID.SSBD feature is enabled (1). The Cross vCenter VMotion now worked.

Next I added an older 6.5u1 host to the cluster and on the newer (6.5u2) host I again ran the vim-cmd and saw that it had changed the CPUID.SSBD value to disabled (0). The Cross vCenter VMotion didn’t work.

When removing the older 6.5u1 host from the cluster the CPUID.SSBD feature didn’t change on the 6.5u2 host, but when I set the EVC to NONE and then changed it back to Westmere, the host had the CPUID.SSBD feature enabled again.

Solved

So the solution to my issue was to just upgrade all the host in the cluster to 6.5u2 and Cross vCenter VMotion worked without issues. Looking back at this issue it is of course logical what happened, but I would have expected somehow that there may have been a better indication that the 6.5u2 EVC differs a little from the same EVC level in 6.5u1.