Hello,
I'm currently trying to get vfio working kvm on 14.04 and I'm having some issues. My questions come in two parts. Before anything, I don't have much experience working with drivers/modules in linux so I would appreciate it if someone could validate/correct my understanding thus of the system so far. Then, I'll list what I've done to troubleshoot and the symptoms of the problem I'm having. If it doesn't solve my issue, hopefully it'll help someone else.
First, here's what I gather so far. There several ways of doing passthrough in QEMU. The most recent and efficient is via VFIO which uses IOMMU (VT-D for Intel). An older method is via pci-assign. Now, hardware can only be used by one driver/module at a time and, as such, we must prevent anything from taking the GPU/PCI so that it may be bound by vfio-pci. The version or nature of the driver (nouveau vs nvidia for instance) doesn't matter as we will let the guest handle it. The only thing that matters is that the driver does not bind our device of interest. So, the general procedure to setting up VFIO passthrough would be:
Now, here's what I've done:
Relevant specs:
After rebooting, I see that the audio and video devices for my GPU are bound by pci_stub. After the bind script, both devices are bound by vfio-pci. This is checked via lspci.
If I leave -vga cirrus on, I get a code 12 after installing drivers in Windows. If I set -vga to none, I get no video output from the GPU. Now, I'm wondering now if it's a limitation of the mobo or the gpu.
Any ideas anyone?
Thanks :D
I'm currently trying to get vfio working kvm on 14.04 and I'm having some issues. My questions come in two parts. Before anything, I don't have much experience working with drivers/modules in linux so I would appreciate it if someone could validate/correct my understanding thus of the system so far. Then, I'll list what I've done to troubleshoot and the symptoms of the problem I'm having. If it doesn't solve my issue, hopefully it'll help someone else.
First, here's what I gather so far. There several ways of doing passthrough in QEMU. The most recent and efficient is via VFIO which uses IOMMU (VT-D for Intel). An older method is via pci-assign. Now, hardware can only be used by one driver/module at a time and, as such, we must prevent anything from taking the GPU/PCI so that it may be bound by vfio-pci. The version or nature of the driver (nouveau vs nvidia for instance) doesn't matter as we will let the guest handle it. The only thing that matters is that the driver does not bind our device of interest. So, the general procedure to setting up VFIO passthrough would be:
1. IOMMU needs to be enabled
2. The appropriate modules must be loaded on boot
3. Drivers/modules for the GPU must not bind the hardware.
This is done in one of two ways. Either you blacklist the module (assuming it wasn't compiled with the kernel) or you prematurely bind the hardware to pci-stub. The latter appears to be a better solution as modules/drivers are shared if you have multiple GPUs from the same manufacturer so you can't really blacklist a driver that may be used by the host (i.e, you're using two GPUs from the same manufacturer and one of them is used for passthrough and the other for the host).
4. Bind vfio-pci to the devices you wish to passthrough
5. Load the appropriate device in qemu by selecting vfio-pci as device and giving it the pci #.
2. The appropriate modules must be loaded on boot
3. Drivers/modules for the GPU must not bind the hardware.
This is done in one of two ways. Either you blacklist the module (assuming it wasn't compiled with the kernel) or you prematurely bind the hardware to pci-stub. The latter appears to be a better solution as modules/drivers are shared if you have multiple GPUs from the same manufacturer so you can't really blacklist a driver that may be used by the host (i.e, you're using two GPUs from the same manufacturer and one of them is used for passthrough and the other for the host).
4. Bind vfio-pci to the devices you wish to passthrough
5. Load the appropriate device in qemu by selecting vfio-pci as device and giving it the pci #.
Now, here's what I've done:
Relevant specs:
Intel 4430
AsRock B85M-ITX
evga GTX750 Ti
Kernel 3.16
QEMU 2.0 or latest from git
SeaBios 1.7.4 or 1.7.5
And the steps:Bios:
Set Vt-d to enable
Set Primary GPU to onboard
1. Enabled IOMMU via /etc/default/grub:
Code:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1"then
Code:
sudo update-grub2. Added relevant modules to /etc/modules
Code:
pci_stubvfiovfio_iommu_type1vfio_pcikvmkvm_intel3. Blacklisted the native drivers via /etc/modprobe.d/blacklist.conf
Code:
blacklist nouveaublacklist nvidiaBind the devices to pci_stub on boot (just in case) in /etc/initramfs-tools/modules
Code:
pci-stub ids=10de:1380,10de:0fbcthen
Code:
sudo update-initramfs -u4. Bound vfio to the devices via:
Code:
#!/bin/bashDEVLIST="0000:01:00.0 0000:01:00.1"for dev in $DEVLISTdo vendor=$(cat /sys/bus/pci/devices/$dev/vendor) device=$(cat /sys/bus/pci/devices/$dev/device) if [ -e /sys/bus/pci/devices/$dev/driver ] then echo $dev > /sys/bus/pci/devices/$dev/driver/unbind fi echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_iddonemodprobe vfio-pci5. Starting QEMU
Code:
#!/bin/bashsudo qemu-system-x86_64 -enable-kvm -M q35 -m 2048 -cpu host \-smp 4,sockets=1,cores=4,threads=1 \-bios /usr/share/qemu/bios.bin -vga cirrus \-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 \-drive file=./W764.qcow2,id=disk,format=qcow2 -device ide-hd,bus=ide.0,drive=disk \-drive file=./W764.iso,id=isocd -device ide-cd,bus=ide.1,drive=isocd \-boot menu=onIf I leave -vga cirrus on, I get a code 12 after installing drivers in Windows. If I set -vga to none, I get no video output from the GPU. Now, I'm wondering now if it's a limitation of the mobo or the gpu.
Any ideas anyone?
Thanks :D