Recently I upgraded to kernel 6.0 on my Manjaro rig, but that didn’t go well. The PC booted but there was no screen output. I was able to ssh into the Manjaro host, though.
Searching for an answer led me to the “Linux Kernel 6 seems to be incompatible with the vfio_pci module needed for PCI passthrough” on Level1Techs. If you go further down the thread, you’ll see how someone posted a working solution “Using the driver_override feature“.
For several years I had used the driver-override feature without a problem and still think it should be the preferred method. However, this feature stopped working with kernel 5.11 (not sure if and when it was fixed in the 5.x kernel). Since then I use the grub method to bind the graphic card and other PCI devices to the vfio-pci driver (vfio-pci.ids=…).
I haven’t tried the driver-override feature with kernel 6 yet, nor did I try the other methods described in my “Blacklisting Graphics Driver” post. Just glad I haven’t deleted or shortened that post. It’s always good to have multiple options.
As of now, I suggest to NOT upgrade to kernel 6, unless you have a real good reason (e.g. new hardware support in kernel 6).
If you tried any of the methods described in my “Blacklisting Graphics Driver” post, please comment on the results below. Thanks!
Edit January 22, 2023: In the meantime I switched to the grub method and am now using the latest 6.1 kernel in the Manjaro Linux tree without issues.
This works well on fedora.
/etc/modprobe.d/vfio.conf:
options vfio-pci disable_vga=1
blacklist nouveau
options nouveau modeset=0
softdep snd_hda_intel pre: vfio-pci
softdep nouveau pre: vfio-pci
softdep nvidia pre: vfio-pci
softdep xhci_hcd pre: vfio-pci
/etc/dracut.conf.d/vfio.conf
dd_drivers+=” vfio vfio_iommu_type1 vfio_pci ”
/etc/defaults/grub
GRUB_CMDLINE_LINUX=”rd.driver.blacklist=nouveau nvidia-drm.modeset=1 initcall_blacklist=simpledrm_platform_driver_init intel_iommu=on iommu=pt vfio-pci.ids=10de:1f08,10de:10f9,10de:1ada,10de:1adb rhgb quiet”
Got this off stack exchange but can’t find the page.
Any news about this one?
I’ve gone a long time with a very well running VM with Windows 10 – I even changed my guest GPU along the way from a Nvidia 1650 GTX to a AMD RX 3700 (only issue there seems to be that I can’t get the new GPU to stick – I have to delete the old and add the new after every reboot, but that’s a minor nuisance).
Until today when I decided to try an upgrade to 6.2 to see if perhaps I could take care of another issue I have since updating to Linux Mint 21.2 (unrelated to Virtualization), but ran into a problem where my host gpu/monitor also goes off (or down?) when starting my VM. Later – after shutting down my VM I have to do a hard reset to get a working monitor again.
I skimmed through the thread you linked to above, but that one kind of goes a bit over my head and I’m struggling to understand what I can do about it.
I always have the option of returning to before 6.2 (long live Timeshift) but I don’t want to throw in the towel just yet.
Hello Göran,
The first thing I would do is downgrade. I currently run my VM on a Manjaro host – a rolling distribution that is considered quite bleeding edge – using the latest stable kernel which is 6.1.51. My Linux Mint media server indeed is on kernel 6.2, but that server doesn’t run any VMs.
There have been some changes recently with regard to how drivers are assigned. I’m currently using the grub method which – surprise – is working fine again. You should check my post on “Blacklisting Graphics Drivers” – see https://www.heiko-sieger.info/blacklisting-graphics-driver – and specifically the relatively new driverctl utility – see https://gitlab.com/driverctl/driverctl for more information and instructions. The driverctl utility can be installed in Linux Mint using
sudo apt install driverctl.When you say that you can’t get the new GPU to stick, where exactly do you need to delete the old and add the new?
Also have a look at my complete tutorial here: https://www.heiko-sieger.info/creating-a-windows-10-vm-on-the-amd-ryzen-9-3900x-using-qemu-4-0-and-vga-passthrough. Never mind the hardware specs I gave, it is pretty universal. It also gives instructions for Ubuntu or Linux Mint (I originally wrote it for Pop-OS, but dumped that distribution due to its problems and lack of support).
Hope you get your system back up and running quickly.
Hi Heiko,
Thanks for replying. And yes – your tutorial you’re referring to was actually the one that got me up and running when I did my first install, and I really liked the way you walk through all steps with plenty of screenshots and very easy to follow instructions. I have even shared that one a couple of times to people – something I usually don’t do unless I find what I’m passing on useful. And what I like the most is that I have everything in one place, instead of having to go fishing all over the net trying to put the pieces together.
About the new GPU – I was talking about virt-manager. I always have to first remove the old hardware (PCI Host Device) and then add the new one manually. Fair enough – I also have to do that with any usb host device I – temporarily – plug in for file transfers, but that makes more sense (as any computer works the same) than having to swap the GPU every time I reboot the host. It’s like the xml was set “fixed” at my initial setup and as I can’t find any “save” button anywhere it just reverts to the state of the first setup at launch.
Regarding my current problem with kernel 6.2 and my host adapter going down (or whatever I should call it) as well whenever I launch my VM… I believe I’m actually already using the grub solution, but that one doesn’t seem to be working for me with kernel 6.2. I’m currently installing driverctl which seems easy enough to set up (too easy?), I’ll see how that one turns out. I suppose I have to comment out the previous solution from grub before going ahead as well?
Anyway – If this doesn’t work out I’ll probably have to roll back the kernel and just leave it be as I have a weekly photo processing commitment that I run in Windows which gives me very little time in between to tinker, but I will try to fight this one out a couple of more days. Besides – I could probably stay on Mint 21.2 with their “default” kernel for quite some time without issues, but once I do decide to upgrade I might find myself in the same mess again, so I figure I’ll try to solve it sooner than later.
I was traveling and couldn’t respond earlier.
1. To list all your VMs (domains), use: sudo virsh list –all
2. You can edit your .xml config file using: sudo virsh edit name_of_vm
3. You will land in a vi or vim-style editor. If you are not familiar with it, it might be a good idea to make yourself familiar with the basics: https://www.redhat.com/sysadmin/get-started-vi-editor
4. Change your graphics card etc.
5. Save changes with :wq
Virtual Machine Manager is a front-end that makes use of virsh to create and edit configurations. I actually quite hate the way how things are obfuscated. There are config files under /etc/libvirt/qemu/ but you can’t edit them because of the way virsh works. That’s why for many years I refused to use virsh / virt-manager but it’s like swimming against the stream.
In my setup everything works and once I edit the configuration in virt-manager (Virtual Machine Manager) the changes are permanent. It might have to do with permissions. Check the following files:
6. /etc/passwd
It should have an entry like: libvirt-qemu:x:953:953:Libvirt QEMU user:/:/usr/bin/nologin
7. /etc/group
This file should have the following entries (the numbers can be different):
libvirt-qemu:x:953:
qemu:x:952:
libvirt:x:966:your_user_name
kvm:x:992:libvirt-qemu,qemu
Make sure your username is a member of the libvirt group.
8. With regard to your changing the way of binding the graphics card from grub to driverctl, make sure to run sudo update-grub after you modify the /etc/default/grub file.
9. There is a very helpful Discord group called VFIO. It might help to post there under kvm-support.
Good luck!