====== Infiniband MLX4 on FreeBSD ======

===== Add the Following to your Kernel and Compile =====
<file>
options        OFED            # Infiniband protocol stack and support
options        SDP             # Sockets Direct Protocol for infiniband
options        IPOIB_CM        # Use connected mode IPoIB

options COMPAT_LINUXKPI
device         ipoib           # IP over IB devices
device         mlx4ib          # ConnectX Infiniband support
device         mlxen           # ConnectX Ethernet support
device         mthca           # Infinihost cards
device mlx5
device mlx5en
</file>

===== Add to or create /etc/src.conf =====
<file>
WITH_OFED='yes'
</file>

===== Using Ethernet mode =====
  * it seems like the mlx4 adapter start by default in ethernet mode which seems the most stable
  * below is a rc.conf for using ethernet mode
  * on 11.1 opensm could not find the adapter in infiniband mode
  * It seems like using 9k jumbo frames yield a significant performance gain
<file>
ifconfig_mlxen0="inet x.x.x.x netmask x.x.x.x mtu 9000"
</file>

===== Add to /boot/loader.conf (optional) =====
<file>
mlx4_load="YES"
ibcore_load="YES"
mlx4ib_load="YES"
ipoib_load="YES"
</file>