User Tools

Site Tools


ubuntu:iser_rdma

This is an old revision of the document!


ISER/RMDA with ZFS on Ubuntu Server 17.04

gg

RDMA Infiniband Modules

/etc/modules
rdma_cm
ib_uverbs
rdma_ucm
ib_ucm
ib_umad
ib_ipoib
 
mlx4_ib
mlx4_en
mlx5_ib
iw_cxgb3
iw_cxgb4
iw_nes
iw_c2
rpcrdma
/etc/security/limits.conf
...
* soft memlock unlimited
* hard memlock unlimited
root soft memlock unlimited
root hard memlock unlimited
...
/etc/network/interfaces
auto ib0
iface ib0 inet static
        address 192.168.113.1
        netmask 255.255.255.0

Comment out discovery command and add a new ISER Command

/usr/share/perl5/PVE/Storage/ISCSIPlugin.pm
#my $cmd = [$ISCSIADM, '--mode', 'discovery', '--type', 'sendtargets', '--portal', $portal];
my $cmd = [$ISCSIADM, '--mode', 'discovery', '--type', 'sendtargets', '--portal', $portal, '-I', 'iser'];

URLS

Other Commands

Check IO Delay/Sats

apt install sysstat
iostat -x 1

Disable ISCSI Authentication

cd iscsi/iqn.*/tpgt1/ 
set attribute authentication=0 demo_mode_write_protect=0 generate_node_acls=1 cache_dynamic_acls=1

Enable ISCSI iSER

cd iscsi/iqn.*/tpgt1/portals/192.168.221.1:3260
enable_iser true

Create File based iSCSI backstore

cd /backstores/fileio/
create file_or_dev=/tank/vol1/iservol2 name=iservol2 size=20G write_back=true

Check ZFS IO Stats

zpool iostat -v 1

Discover Targets from ISCSI Portal

iscsiadm -m discovery -p <IP>:<PORT>

Refresh LUNS of ISCSI Target

iscsiadm -m node -R

Logout of all Targets

iscsiadm -m node -u

Logout to all Targets

iscsiadm -m node -l

Logout of a Target

iscsiadm -m node -T <TARGETNAME> -u

Logout to a Target

iscsiadm -m node -T <TARGETNAME> -l
ubuntu/iser_rdma.1519238252.txt.gz · Last modified: 2018/02/21 12:37 by tschulz