After a lot of fiddling around with settings and hardware, I finally have a stable Proxmox 5.1 ATI GPU pass-through system. What helped me was this helpful article to finally get all the bugs ironed out. I did have to make several tweaks for my system. I’m running an Intel system with ATI Radeon GPU.
- Ensure VT-d is supported and enabled in the BIOS
- Enable IOMMU on the host
- append the following to the GRUB_CMDLINE_LINUX_DEFAULT line in /etc/default/grub
intel_iommu=on
- Save your changes by running
update-grub
- Blacklist NVIDIA & Nouveau kernel modules so they don’t get loaded at boot
-
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
echo "blacklist radeon" >>
/etc/modprobe.d/blacklist.conf
- Save your changes by running
update-initramfs -u
- Add the following lines to /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
- Reboot the host
- Create your Windows VM using the UEFI bios hardware option (not the deafoult seabios) but do not start it yet. Use VirtIO. Modify /etc/pve/qemu-server/<vmid>.conf and ensure the following are in the file. Create / modify existing entries as necessary.
bios: ovmf
machine: q35
cpu: host,hidden=1
numa: 1
- Install Windows
- Mount second ISO (virtio-win*.iso)
- Load IO driver from d:\viostore\w10\amd64\viostore.inf
- After install be sure to enable Remote desktop.
- Pass through the GPU.
- Modify /etc/pve/qemu-server/<vmid>.conf and add
hostpci0: <device address>,x-vga=on,pcie=1. Example
hostpci0: 01:00,x-vga=on,pcie=1
- Passthrough USB keyboard and mouse
- I find it best to passthrough specific USB ports rather than device IDs. That way I can hotplug different devices to specific ports later without having to reboot.
- Done.
Troubleshooting
Blue screening when launching certain applications
AMD drivers setup application and/or Windows boot would consistently blue screen on me with the following error:
kmode_exception_not_handled
The fix as outlined here was to create /etc/modprobe.d/kvm.conf and add the parameter “options kvm ignore_msrs=1”
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
Same fix can be applied at runtime with
echo 1 > /sys/module/kvm/parameters/ignore_msrs
Update 4/9/18: Blue screening happens to Windows 10 1803 as well with the error
System Thread Exception Not Handled
The fix for this is the same – ignore_msrs=1
Frezing keyboard/mouse:
Device Manager -> Human Interface Devices -> Microsoft Hardware USB Keyboard -> Power Management -> Uncheck “Allow the computer to turn off this device to save power”
Notes:
Other guides require setting up vfio.conf. With my hardware it was not required. It’s probably needed for a nVidia card though.
- Determine the PCI address of your GPU
- Run
lspci -v
and look for your card. Usually 01:00.0 & 01:00.1. You can omit the part after the decimal to include them both in one go – so in that case it would be 01:00
- Run lspci -n -s <PCI address> to obtain vendor IDs. Example :
lspci -n -s 01:00
01:00.0 0300: 10de:1b81 (rev a1)
01:00.1 0403: 10de:10f0 (rev a1)
- Assign your GPU to vfio driver using the IDs obtained above. Example:
echo "options vfio-pci ids=10de:1b81,10de:10f0" > /etc/modprobe.d/vfio.conf