ivoras’ FreeBSD blog

January 6, 2008

Encrypted iSCSI target

Filed under: FreeBSD — ivoras @ 12:50 am

Suppose you want to use a remote iSCSI device, but you don’t exactly trust either the storage or the network in between. Of course, there’s a way around it :)

The setup presented here is very simple and will behave like this:

[iSCSI server] -- encrypted data on the server and over the wire -- [iSCSI client]


Note: all these instructions are valid for FreeBSD 7.0 – previous versions are probably missing some parts.

Setting up an iSCSI target

You can skip this section if you already have an iSCSI target (a “target” is where the data is stored, i.e. the “server” node of iSCSI).

1. Install the iscsi-target port.

2. Edit /usr/local/etc/iscsi/targets file and add lines similar to the following:

# NAME          DEVICE          START           LENGTH
extent0         /dev/da2        0               5GB
# NAME          ACCESS          STORAGE         NETMASK
target0         rw              extent0         10.0.0.0/24

These lines should be self-explanatory. If you need more help, see targets(5) or NetBSD’s iscsi-target HOWTO.

3. Enable iscsi-target in /etc/rc.conf by adding the following line to it:

iscsi_target_enable="YES"

4. Start the server by running /usr/local/etc/rc.d/iscsi_target start. You should see something like the following outputted to the console:

Starting iscsi_target.
Reading configuration from `/usr/local/etc/iscsi/targets'
target0:rw:10.0.0.0/24
        extent0:/dev/da2:0:5368709120
DISK: 1 logical unit (10485760 blocks, 512 bytes/block), type iscsi fs
DISK: LUN 0: 5120 MB disk storage for "target0"
TARGET: TargetName is iqn.1994-04.org.netbsd.iscsi-target

Setting up the iSCSI initiator

The “initiator” is the client part in iSCSI, and it connects to the server. The following steps should be done on the client system.

1. Edit /etc/iscsi.conf and add the following lines:

target0 { # nickname
        targetaddress        = 10.0.0.102
        targetname           = iqn.1994-04.org.netbsd.iscsi-target:target0
}

2. Load the iscsi_initiator kernel module with:

# kldload iscsi_initiator

Also, add the following line to /etc/loader.conf to load the module on boot:

iscsi_initiator_load="YES"

3. Start the iSCSI session by running:

# iscontrol -n target0

Several lines should be output to the console, which should look like the following:

iscontrol[8516]: running
iscontrol[8516]: (pass3:iscsi0:0:0:0):  tagged openings now 0
iscontrol[8516]: cam_open_btl: no passthrough device found at 1:0:1
iscontrol[8516]: cam_open_btl: no passthrough device found at 1:0:2
iscontrol[8516]: cam_open_btl: no passthrough device found at 1:0:3
iscontrol: supervise starting main loop

More importantly, the kernel log (which you can see with tail /var/log/messages) should now contain something similar to this output:

Jan  4 23:17:08 client kernel: da0 at iscsi0 bus 0 target 0 lun 0
Jan  4 23:17:08 client kernel: da0:  Fixed Direct Access SCSI-3 device

This means the device da0 has been created – this is the local representation of the remote iSCSI drive.

3. Set up GEOM_GELI on the new device:

# geli init /dev/da0

The utility will ask you for a passphrase which will be used to encrypt the data. GEOM_ELI (as is the encryption layer known) has many more options, but the defaults are good enough. It will use AES encryption with sane defaults.

4. Load the GEOM_ELI kernel module:

# kldload geom_eli.ko

Also, add the following to /boot/loader.conf:

geom_eli_load="YES"

5. Attach the encrypted device:

# geli attach /dev/da0

Lines similar to the following should appear in the kernel log:

Jan  4 23:33:28 client kernel: GEOM_ELI: Device da0.eli created.
Jan  4 23:33:28 client kernel: GEOM_ELI: Encryption: AES-CBC 128
Jan  4 23:33:28 client kernel: GEOM_ELI:     Crypto: software

The device da0.eli has been created – this is the end-point device that can be used by file systems and for other purposes (swap, etc.).

6. Make the file system and mount it!

# newfs -U -L mydata /dev/da0.eli

A successful run of newfs looks something like this:

/dev/da0.eli: 5120.0MB (10485756 sectors) block size 16384, fragment size 2048
        using 28 cylinder groups of 183.77MB, 11761 blks, 23552 inodes.
        with soft updates
super-block backups (for fsck -b #) at:
 160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624, 3010976, 3387328,
 3763680, 4140032, 4516384, 4892736, 5269088, 5645440, 6021792, 6398144,
 6774496, 7150848, 7527200, 7903552, 8279904, 8656256, 9032608, 9408960, 9785312, 
 10161664

Since we used a volume label for the file system, observe the following message in the kernel log:

Jan  4 23:38:17 client kernel: GEOM_LABEL: Label for provider da0.eli is ufs/mydata.

Now you can mount the file system:

# mount /dev/ufs/mydata /mydata

And that’s it!

There are two points that can’t be readily automated right now: the iscontrol step which starts the iSCSI initiator, and the geli requiring a password. The former can be approximated by creating a small shell script that does the step and putting it in /usr/local/etc/rc.d but the second cannot be, since the whole point of having an encrypted storage is that it isn’t accessible by unwanted people.

The way this setup works is that the unencrypted data is used by the file system (as it should – you wouldn’t be able to use it otherwise) via the da0.eli device. This data is encrypted and the encrypted data is written to da0 device. This is the iSCSI client device and the data is tranferred to the server in its encrypted form. The server and the network never see unencrypted data.

Due to GEOM’s modularity, other components could be added to the data processing graph, such as journaling (gjournal), caching (gcache), etc. in which case the end-point device name will “grow” suffixes, such as da0.eli.journal. Even RAID levels can be added, though it makes little sense to do it on the client (it’s perfectly fine on the server).

3 Comments

  1. fantastic how to Ivan – very helpful and the solid FreeBSD tools simply worked without any problems ….

    I also learned about stackable geom! /dev/da0.eli.journal is cool!

    beyond the obvious geom classes (geli, gjournal) are there any neat tips you could offer about how to hook iscsi together with gvirstor? :-)

    Comment by ooga — March 10, 2008 @ 6:14 am

  2. Hi Ivan.

    Have you ever been able to mount an iscsi target from a Linux initiator?

    So far I have not been able to accomplish this.

    – Zog

    Comment by Mr Zog — November 11, 2008 @ 2:42 am

  3. I’ve never tried connecting from Linux. Try to look around for documentation on the NetBSD’s iSCSI target (this is the one used in FreeBSD).

    Comment by ivoras — November 11, 2008 @ 4:40 pm

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress