Preface - I've run GPU passthrough on VMware for a couple of years using a GTX480 softmodded into a Quadro 6000. But the nVidia driver crippling (-cpu kvm=off) really bothered me so I swore I would go AMD if they had something even halfway decent. I liked what I saw in the benches of the RX4xx so I grabbed a Radeon RX460 and have spent the last few days trying to pass it through to KVM.
My hardware is:
It's disappointing the E3 Xeons don't support ACS. No IGD is fine.
My /etc/default/grub
My /etc/modprobe.d/kvm-iommu.conf
My /etc/modprobe.d/vfio.conf
These match by RX460 and RX460 Sound. Apparently disable_vga=1 is only for OVMF .fd, I tried without this using seabios.bin as well
My /etc/modprobe.d/blacklist.conf
My /etc/initramfs-tools/modules
These match my HD6350, RX460 and RX460 Sound
Then updates:
My IOMMU group 1 looks like this:
I'm unclear what this should look like after the pcie_acs_override=downstream, I was understanding this should have split the groups of 01:00.0 (the HD6350) and the 02:00.0 and .1 (RX460)
or according to virsh nodevdev-list --tree
To start the VM I detach the devices from the host and bind the VFIO
vfio bind is this (I take no credit, but can't remember who should get it):
And finally I'm launching qemu-system-x86_64 with variations on:
qemu.log only shows:
a bunch of times
Any attempt to use an x-vga option fails with:
I thought x-vga was necessary. The results vary without it. I can get "console" splash that proceeds to blind mode. I have got it to display on the 2nd monitor a couple times before I had USB but I feel like I'm doing something wrong. Keyboard works, haven't got to anything that uses mouse yet but I assume it's ok.
I've been trying this on both an empty qcow2 disk and a raw disk with 1604 desktop x64 installed and the AMDGPU pro driver already installed. I've been avoiding virt-install or virt-manager until this is working, trying not to jump between methods.
Does anyone have any suggestions or simplifications that might help? Anything conceptually wrong with this?
Thanks.
My hardware is:
Code:
Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz
ASRock Z87 Extreme4
32GB
Radeon HD6350
Radeon RX460My /etc/default/grub
Code:
quiet splash intel_iommu=on iommu=pt iommu=1 pcie_acs_override=downstreamCode:
options kvm vfio_iommu_type1Code:
vfio-pci ids=1002:67ef,1002:aae0 disable_vga=1My /etc/modprobe.d/blacklist.conf
Code:
blacklist radeonCode:
pci_stub ids=1002:67ef,1002:aae0,1002:68f9Then updates:
Code:
update-grub
update-initramfs -uCode:
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.1
/sys/kernel/iommu_groups/1/devices/0000:01:00.0
/sys/kernel/iommu_groups/1/devices/0000:02:00.0
/sys/kernel/iommu_groups/1/devices/0000:02:00.1or according to virsh nodevdev-list --tree
Code:
computer
|
+- pci_0000_00_00_0
+- pci_0000_00_01_0
| |
| +- pci_0000_01_00_0
|
+- pci_0000_00_01_1
| |
| +- pci_0000_02_00_0
| +- pci_0000_02_00_1Code:
virsh nodedev-dettach pci_0000_01_00_0
virsh nodedev-dettach pci_0000_02_00_0
virsh nodedev-dettach pci_0000_02_00_1
vfio-bind 0000:01:00.0
vfio-bind 0000:02:00.0
vfio-bind 0000:02:00.1Code:
#!/bin/bash
modprobe vfio-pci
for dev in "$@"; do
vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
device=$(cat /sys/bus/pci/devices/$dev/device)
if [ -e /sys/bus/pci/devices/$dev/driver ]; then
echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
fi
echo $vendor $device | tee /sys/bus/pci/drivers/vfio-pci/new_id
doneCode:
qemu-system-x86_64 \
-enable-kvm \
-M q35 \
-m 4096 \
-cpu host \
-smp 4,sockets=1,cores=4,threads=1 \
-bios /usr/share/qemu/OVMF.fd \
-vga none \
-nographic \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=02:00.0,multifunction=on,rombar=0,id=rx460 \
-device vfio-pci,host=02:00.1,id=rx460snd \
-drive file=/var/lib/libvirt/images/u1604dt03.img,id=disk,format=raw,if=none,cache=writeback \
-drive file=/var/lib/libvirt/images/ubuntu-16.04-desktop-amd64.iso,id=isocd2 \
-usb -device usb-host,hostbus=3,hostaddr=2 -device usb-host,hostbus=3,hostaddr=3 \
-d guest_errors,mmu -D /var/log/qemu.logCode:
CR0 update: CR0=0x60000010Any attempt to use an x-vga option fails with:
Code:
qemu-system-x86_64: -device vfio-pci,host=02:00.0,multifunction=on,rombar=0,id=rx460,x-vga=on: vfio: Device does not support requested feature x-vga
qemu-system-x86_64: -device vfio-pci,host=02:00.0,multifunction=on,rombar=0,id=rx460,x-vga=on: Device initialization failedI've been trying this on both an empty qcow2 disk and a raw disk with 1604 desktop x64 installed and the AMDGPU pro driver already installed. I've been avoiding virt-install or virt-manager until this is working, trying not to jump between methods.
Does anyone have any suggestions or simplifications that might help? Anything conceptually wrong with this?
Thanks.