User Tools

Site Tools


virt:zfs_kvm_debian8

ZFS and VFIO PCI Passthrough on Debian 8

Config Files

Add intel_iommu=on to the boot loader

/etc/defaults/grub
..
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
..

Load vfio on bootup

/etc/modules
...
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
...

Blacklist Video Drivers

  • If a driver is using a pci device that you want to passthough you might want to blacklist the driver, although you can try unbinding the driver before you tell vfio to claim the device
  • driver name is found be running:
    lspci -vn | less -p VGA

    Look for “Kernel driver in use:

    /etc/modprobe.d/blacklist.conf
    ...
    blacklist mgag200
    blacklist nouveau
    ...

Setup Bridged Networking

/etc/network/interfaces
...
auto eth0
iface eth0 inet manual
 
auto br0
iface br0 inet dhcp
        bridge_ports eth0
...

Install Kernel and updates from Backports

  • add the following line to /etc/apt/sources.list
    deb http://ftp.debian.org/debian jessie-backports main
  • Update package list and update packages
    apt-get update
    apt-get -t jessie-backports upgrade

Build and Install ZFS on Linux (deb method)

  • Download current release from https://github.com/zfsonlinux/zfs/releases. You will need spl-x.x.x.x.tar.gz and zfs-x.x.x.x.tar.gz
  • Install dependency packages
    apt-get install build-essential gawk alien fakeroot linux-headers-$(uname -r) zlib1g-dev uuid-dev libblkid-dev libselinux-dev parted lsscsi
  • Build and install spl
    tar xvzf spl-x.x.x.x.tar.gz
    cd spl-x.x.x.x
    ./configure
    make deb-utils deb-kmod
    dpkg -i *.deb
  • Build and install zfs
    tar xvzf zfs-x.x.x.x.tar.gz
    cd zfs-x.x.x.x
    ./configure
    make deb-utils deb-kmod
    dpkg -i *.deb

Bind PCI device to VFIO

  1. To grab the hexadecimal PCI id. run
    lspci -nv
  2. e.g. for
    07:00.0 0300: 10de:0322 (rev a1) (prog-if 00 [VGA controller])

    the PCI ID would be 10de:0322. The slot ID/address is 07:00.0

  3. Then take the slot address in the previous example *07:00.0 and prepend “0000:” so that you have something like 0000:07:00.0. Then execute the following command to unbind the current driver
    echo 0000:07:00.0 > /sys/bus/pci/devices/0000:07:00.0/driver/unbind
  4. Then take the PCI ID in the previous example 10de:0322 and replace the “:” with a space then run the following command
    echo 102b 0522 > /sys/bus/pci/drivers/vfio-pci/new_id

USB Passthough

  • If you are passing a physical video card to a VM, you will need to also pass usb input devices to the VM since the VM will not have a SDL or Spice virtual display.
  • This is done through USB passthrough.
  • To find the USB id's to pass through run
    lsusb | grep -v "root hub"
  • This will give you something like this
    Bus 005 Device 002: ID 046b:ff10 American Megatrends, Inc. Virtual Keyboard and Mouse
    Bus 003 Device 003: ID 0557:2205 ATEN International Co., Ltd 
    Bus 003 Device 002: ID 0451:2046 Texas Instruments, Inc. TUSB2046 Hub
  • What we want is the hexadecimal ID ffff:ffff
  • then this to pass the usb device to the vm
    -usbdevice host:ffff:ffff

Example kvm script

#!/bin/sh
 
echo 0000:07:00.0 > /sys/bus/pci/devices/0000:07:00.0/driver/unbind
echo 102b 0522 > /sys/bus/pci/drivers/vfio-pci/new_id
 
qemu-system-x86_64 -enable-kvm -M q35 -m 2048 \
-cpu host -smp 4,sockets=1,cores=4,threads=1 \
-net nic,model=virtio,vlan=0 -net tap,vlan=0,ifname=tap0,script=/etc/qemu-ifup \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-drive file=/vol1/iso/bie2k8r2411.iso,index=2,media=cdrom \
-drive file=/vol1/iso/virtio-win.iso,index=3,media=cdrom \
-drive file=/vol1/hd-img/win2kr3.qed,format=qed,if=virtio,cache=writeback,index=1 \
-usb -usbdevice host:0557:2205 \
-boot menu=on \
-vga none

Important Bits

  • The following is used to tell kvm to use the pci video card for display not a SDL or Spice canvas
    -vga none
  • The following is used to tell kvm to setup bridged networking
    -net nic,model=virtio,vlan=0 -net tap,vlan=0,ifname=tap0,script=/etc/qemu-ifup
  • The following is used to add a pcie bridge to the vm (needed for pci passthrough)
    -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1
  • The following adds ICH9 emulation that enables smoother pci passthrough
    -M q35
  • The following unbinds the current driver and tells vfio to claim the pci device
    echo 0000:07:00.0 > /sys/bus/pci/devices/0000:07:00.0/driver/unbind
    echo 102b 0522 > /sys/bus/pci/drivers/vfio-pci/new_id
  • The following passes a video card to the vm and binds it to the pcie bridge
    -device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
  • The following passes a usb hub to the vm so that keyboard and mouse input can be used with the physical display
    -usbdevice host:0557:2205
  • Add Spice Display
    -vga qxl -spice port=5900,addr=127.0.0.1,disable-ticketing
virt/zfs_kvm_debian8.txt · Last modified: 2016/04/20 14:18 by tschulz