Nvidia And The “hidden state”

Ever since I started to run a Microsoft Windows VM with GPU passthrough, Nvidia graphics drivers would only support their professional Quadro line of graphics cards in a virtual machine. Ten years ago I bit the bullet and bought an outrageously expensive Nvidia Quadro 2000 GPU. Truth be told – it’s been and still is a great GPU and I currently use it for my Linux host. Back then the Quadro was passed through to a Windows 7 VM running on Xen. It worked great.

Some time later smart developers found a workaround for QEMU/kvm: they were simply hiding some tell-tail signs of the virtual machine. Upgrading to Windows 10 and a new Nvidia GTX 970 GPU, I gave kvm a try. Again, the benchmarks and performance were right on target.

Nvidia Workaround

Nvidia graphics drivers would test to see if the graphics card runs in a virtual machine. Then, depending on the GPU model you owned, it would either work (professional line of Nvidia GPUs) or quit.

Using QEMU/kvm this required you to manually add the following lines to your VM configuration xml:

<kvm>
   <hidden state="on"/>
</kvm>

The above option hides the kvm state from the virtual machine.

Many users including me use the virt-manager GUI to configure and start the VM. Yet, for some inexplicable reason the developers never managed to add the “hidden state=on” option into their menus and GUI.

As a result, the Internet was flooded with “advise” that Nvidia graphics cards are no good for vfio. Especially new vfio / GPU passthrough users were intimidated by the confusion, uncertainty and/or the need to “hack” the VM configuration file.

AMD Can Do

AMD, on the other hand, didn’t have this issue. Their drivers didn’t care if they run on bare metal machines or in a virtual machine. But many AMD GPUs had issues with properly resetting the GPU (“function level reset” or FLR), until the latest RX 6000 series of GPU that fixed it.

Here too a smart developer found a way to reset the GPU. It turned out that AMD’s FLR bug was a lot more troublesome and took quite some effort to work around. Yet, many vfio newcomers thought that AMD was the better, or even only option.

While AMD finally fixed the FLR bug, they recently introduced their own check in their drivers. This means that now AMD requires the vendor id spoofing that was previously only needed with Nvidia.

Another caveat with the new 6000 line of GPUs: CSM (see your BIOS settings) is not supported, it will hang the GPU when exiting the VM.

New Nvidia Drivers

Finally Nvidia showed some sense. With one of the driver updates, Nvidia dropped the VM check and happily plays along with virtual machines. In other words – no more need to add the <hidden state="on"/> option.

Although this has been posted on forums some months ago, I just recently bothered to test it on my PC. What can I say – it works.

That is good news and we can hope that Nvidia has learned that you don’t punish your customers by implementing ridiculous limitations.

For a detailed description on how to pass through a modern Nvidia GPU to a Windows (or other) guest, see my tutorial “Passing Through a Nvidia RTX 2070 Super GPU“.

If this article has been helpful, click the “Like” button below. Don’t forget to share this page with your friends.