There really isn’t any recommendation for disabling or enabling Hyper-Threading for HyperV hosts. The general consensus seems to be, it doesn’t hurt performance, so it’s better to leave it on. We did leave it on, and saw something weird. We had a 20 real core host with Hyper-Threading enabled, so it shows up having 40 cores in windows. The screenshot below is from a 24 real core system:
It shows 24 real cores with 48 logical processors. In HyperV, the number of vCPU’s doesn’t mean the number of real cores the virtual machine gets to use. But it represents the total allowed CPU time of the host, the guest is allowed to use. First, a bit more information on this topic. This percentage can be seen in HyperV manager:
with 1 vCPU, this guest is allowed to use 5% of the hosts cpu resources, with 2vCPU’s:
It’s allowed to use 10% of the hosts CPU resources. This makes sense, since HyperV doesn’t bind the vCPU’s to physical CPU’s, but schedules the threads of the guest to a real CPU, and doesn’t allow those threads to use more than the percentage of total CPU time. To my knowledge, HyperV calculates the percentage per vCPU: 1 vCPU percentage = 100/Number of cores in the host. In the example above, this is 5%, since is has 20 cores.
On a host with 20 real cores and HT enabled, the percentage/vCPU is 2.5%, and this is where it gets interesting, because those 40 cores, aren’t real 40 cores. I was under the impression, that HyperV was HT aware, and therefore should know the difference between the number of real and logical cores. In our case. We had a host with 15 virtual machines, of which, only 5 are in production at all times, the other 10 remain idle. We gave each machine 6 vCPU’s which means that each guest is allowed up to 15% total CPU time of the host, with a maximum of 75% total host CPU usage.
When in production, we never saw the hosts CPU usage go above 35%. None of the Virtual Machines with load saw a CPU usage over 6%. But we were experiencing stalls, and CPU performance issues on the virtual machines on that host.
This was strange, and it looked as if the hypervisor was having issues with scheduling the threads to the correct cores (with the lowest load). So we disabled Hyper-Threading on the host, and gave each virtual machine, 3 vCPU’s. This allowed each virtual machine to use the same 15% of total CPU resources. Now, the hosts sees a highest CPU utilization of 65%. And the guests have a max CPU usage of about 14%. That’s more like it, but apart from the higher utilization of the host and guests. The Effect was that the weird stalls in the guest machines were gone. It was a much smoother experience.
So, where does this leave us with Hyper-Threading on HyperV hosts? There really isn’t a lot of information about how the HyperV scheduler really works. Since the general consensus was that Hyper-Threading shouldn’t hurt HyperV performance. Combined with the fact that Windows Server 2012R2 is Hyper-Threading aware. The assumption was that the scheduler would be smart enough to take Hyper-threaded cores in account. As this case has shown, this isn’t the case. In fact, Hyper-Threading can actually really hurt you’re overall performance on a HyperV host. In the discussed situation, the virtual machines were all remote desktop hosts. With lot’s of users and specific applications, and the load tends to be quite high compered to other scenario’s.
As it makes sense that it’s really hard for a hypervisor scheduler to correctly schedule threads to an already virtualized CPU core. Especially if the host has a high workload. I would recommend to turn off Hyper-Threading on HyperV hosts. It worked out quite well for this scenario, and I can imagine, that in other situations where the host is under heavy load, the scheduling would also be an issue with Hyper-threading enabled.