====== ZFS and VFIO PCI Passthrough on Debian 8  ======

===== Config Files =====

==== Add intel_iommu=on to the boot loader ====

<file bash /etc/defaults/grub>
..
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
..
</file>

==== Load vfio on bootup ====
<file bash /etc/modules>
...
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
...
</file>

==== Blacklist Video Drivers====
  * If a driver is using a pci device that you want to passthough you might want to blacklist the driver, although you can try unbinding the driver before you tell vfio to claim the device
  * driver name is found be running: <file>lspci -vn | less -p VGA
</file>  Look for "**Kernel driver in use:**"<file bash /etc/modprobe.d/blacklist.conf>
...
blacklist mgag200
blacklist nouveau
...
</file>

==== Setup Bridged Networking ====
<file bash /etc/network/interfaces>
...
auto eth0
iface eth0 inet manual

auto br0
iface br0 inet dhcp
        bridge_ports eth0
...
</file>

===== Install Kernel and updates from Backports =====
  * add the following line to /etc/apt/sources.list<file>deb http://ftp.debian.org/debian jessie-backports main</file>
  * Update package list and update packages<file>apt-get update
apt-get -t jessie-backports upgrade</file>

===== Build and Install ZFS on Linux (deb method) =====
  * Download current release from [[https://github.com/zfsonlinux/zfs/releases]].  You will need **spl-x.x.x.x.tar.gz** and **zfs-x.x.x.x.tar.gz**
  * Install dependency packages<file>apt-get install build-essential gawk alien fakeroot linux-headers-$(uname -r) zlib1g-dev uuid-dev libblkid-dev libselinux-dev parted lsscsi
</file>
  * Build and install spl <file>tar xvzf spl-x.x.x.x.tar.gz
cd spl-x.x.x.x
./configure
make deb-utils deb-kmod
dpkg -i *.deb</file>
  * Build and install zfs <file>tar xvzf zfs-x.x.x.x.tar.gz
cd zfs-x.x.x.x
./configure
make deb-utils deb-kmod
dpkg -i *.deb</file>
===== Bind PCI device to VFIO =====
  - To grab the hexadecimal PCI id. run<file>lspci -nv</file>
  - e.g. for <file>
07:00.0 0300: 10de:0322 (rev a1) (prog-if 00 [VGA controller])</file> the PCI ID would be **10de:0322**.  The slot ID/address is **07:00.0**
  - Then take the slot address in the previous example ***07:00.0** and prepend "0000:" so that you have something like **0000:07:00.0**.  Then execute the following command to unbind the current driver<file>
echo 0000:07:00.0 > /sys/bus/pci/devices/0000:07:00.0/driver/unbind
</file>
  - Then take the PCI ID in the previous example **10de:0322** and replace the ":" with a space then run the following command <file>
echo 102b 0522 > /sys/bus/pci/drivers/vfio-pci/new_id
</file>

===== USB Passthough =====
  * If you are passing a physical video card to a VM, you will need to also pass usb input devices to the VM since the VM will not have a SDL or Spice virtual display.
  * This is done through USB passthrough.
  * To find the USB id's to pass through run <file>lsusb | grep -v "root hub"</file>
  * This will give you something like this<file>Bus 005 Device 002: ID 046b:ff10 American Megatrends, Inc. Virtual Keyboard and Mouse
Bus 003 Device 003: ID 0557:2205 ATEN International Co., Ltd 
Bus 003 Device 002: ID 0451:2046 Texas Instruments, Inc. TUSB2046 Hub</file>
  * What we want is the hexadecimal ID **ffff:ffff**
  * then this to pass the usb device to the vm<file>-usbdevice host:ffff:ffff</file>
===== Example kvm script =====
<file bash>
#!/bin/sh

echo 0000:07:00.0 > /sys/bus/pci/devices/0000:07:00.0/driver/unbind
echo 102b 0522 > /sys/bus/pci/drivers/vfio-pci/new_id

qemu-system-x86_64 -enable-kvm -M q35 -m 2048 \
-cpu host -smp 4,sockets=1,cores=4,threads=1 \
-net nic,model=virtio,vlan=0 -net tap,vlan=0,ifname=tap0,script=/etc/qemu-ifup \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-drive file=/vol1/iso/bie2k8r2411.iso,index=2,media=cdrom \
-drive file=/vol1/iso/virtio-win.iso,index=3,media=cdrom \
-drive file=/vol1/hd-img/win2kr3.qed,format=qed,if=virtio,cache=writeback,index=1 \
-usb -usbdevice host:0557:2205 \
-boot menu=on \
-vga none
</file>

==== Important Bits ====
  *  The following is used to tell kvm to use the pci video card for display not a SDL or Spice canvas<file>-vga none</file>
  * The following is used to tell kvm to setup bridged networking<file>-net nic,model=virtio,vlan=0 -net tap,vlan=0,ifname=tap0,script=/etc/qemu-ifup</file>
  * The following is used to add a pcie bridge to the vm (needed for pci passthrough)<file>-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1</file>
  * The following adds ICH9 emulation that enables smoother pci passthrough<file>-M q35</file>
  * The following unbinds the current driver and tells vfio to claim the pci device<file>echo 0000:07:00.0 > /sys/bus/pci/devices/0000:07:00.0/driver/unbind
echo 102b 0522 > /sys/bus/pci/drivers/vfio-pci/new_id</file>
  * The following passes a video card to the vm and binds it to the pcie bridge<file>-device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \</file>
  * The following passes a usb hub to the vm so that keyboard and mouse input can be used with the physical display<file>-usbdevice host:0557:2205</file>
  * Add Spice Display<file>-vga qxl -spice port=5900,addr=127.0.0.1,disable-ticketing</file>