This document is a quickstart guide for a user connected to a Fibre Channel
Forwarder (FCF) and not a SW target. It covers the configuration of the kernel
and installation and configuration of Data Center Bridging (DCB), the HBA API
wrapper library, libhbalinux and fcoe-utils.
This document was written using Fedora 11 as the installed operating system.
The instructions may need adjustments for them to work on other
distributions.
##
# Dependencies
###############
git://open-fcoe.org/libHBAAPI.git
git://open-fcoe.org/libhbalinux.git
git://open-fcoe.org/fcoe-utils.git
##
# Kernel Configuration and Compilation
######################################
1) Download kernel source
There are a number of good choices regarding the most appropriate
kernel source for your needs. fcoe-next.git is a repository that may be
unstable, but will have the latest code. Downloading a released kernel
from kernel.org will give you the most stable kernel, but you'll need to
get user space code that matches the kernel version you've chosen. You can
get user space code that matches stable kernels on the Open-FCoE.org
"Downloads" page or you can pull directly from the git repositories using
tags.
2) Configure the kernel
# make menuconfig
Select the following:
Networking Support -> Networking Options
Data Center Bridging
802.1Q VLAN Support
GVRP (GARP VLAN Registration Protocol) support
Networking Support -> Networking Options -> QoS and/or fair queuing
Multi Band Priority Queueing (PRIO)
Hardware Multiqueue-aware Multi Band Queuing (MULTIQ)
Elementary classification (BASIC)
Universal 32bit comparisons w/ hashing (U32)
Extended Matches
U32 key
Actions
SKB Editing
Device Drivers -> SCSI Device Support -> SCSI Low-level drivers
LibFC
LibFCoE
FCoE
Enable the block layer
Block layer SG support v4
Device Drivers -> Network Device Support -> Ethernet (10000 Mbit)
Intel(R) 10GbE PCI Express adapters support
Data Center Bridging (DCB) Support
[ Replace with desired driver if not using Intel adapter. ]
Exit Saving Changes
3) Compile the kernel
# make && make modules_install && make install
If you're going to build fcoe-utils, you can run 'make headers_install'
at this stage to skip a later step.
4) Configure Grub
Change the default kernel so that this new kernel is used when booting
# reboot
##
# DCB
#######
DEPENDENCIES
* libnl
* libnl-devel
* libconfig
* libconfig-devel
* autoconf
* autotools
* sysconftool
* automake
* libtool
* gcc-c++
* flex
* readline-devel
PROCESS
1) Obtain the latest source
git clone git://open-lldp.org/open-lldp
(note: fcoe-utils versions >= 1.0.20 require an lldpad version >= 0.9.43)
2) Obtain libconfig version 1.3.2 or greater included in distribution
devel packages source can be obtained directly from
http://www.hyperrealm.com/libconfig/
3) Build and install libconfig -- see libconfig documentation
4) Obtain and install devel netlink library (libnl) version 1.1 or greater
included in distribution devel packages source can be obtained directly
from http://www.infradead.org/~tgr/libnl/
5) Bootstrap, configure, make and make install
# cd open-lldp
# ./bootstrap.sh
# rpm --eval "%configure" | sh
# make
# make install
##
# HBA API Wrapper Library
##########################
DEPENDENCIES
* autoconf
* autotools
* sysconftool
* automake
* libtool
PROCESS
1) Obtain the latest source
git clone git://open-fcoe.org/libHBAAPI.git
2) Bootstrap, configure, make and make install
# cd libHBAAPI
# ./bootstrap.sh
# rpm --eval "%configure" | sh
# make
# make install
##
# libhbalinux
##############
DEPENDENCIES
* HBA API Wrapper
* autoconf
* autotools
* sysconftool
* automake
* libtool
* libpciaccess-devel
PROCESS
1) Obtain the latest source
git clone git://open-fcoe.org/libhbalinux.git
2) Bootstrap, configure, make and make install
# cd libhbalinux
# ./bootstrap.sh
# rpm --eval "%configure" | sh
# make
# make install
##
# fcoe-utils
############
DEPENDENCIES
* libnl (see DCB section)
* libhbalinux
* lldpad v0.9.26 or greater (optional configure flag to build without)
* autoconf
* autotools
* sysconftool
* automake
* libtool
PROCESS
1) Obtain the latest source
git clone git://open-fcoe.org/fcoe-utils.git
2) Bootstrap, configure, make and make install
# cd fcoe-utils
# ./bootstrap.sh
Either execute the next line for System V init script install
# rpm --eval "%configure" | sh
or the following line to install systemd unit scripts
# rpm --eval "%configure --with-systemdsystemunitdir=/" | sh
# make
# make install
##
# Example Configuration
#######################
This example configures interface eth3 to automatically connect to storage over
a discovered VLAN.
1) Configure FCoE on the interface
# cd /etc/fcoe/
# cp cfg-ethx cfg-eth3
2) Start lldpad and configure the interface for DCB.
# service lldpad start
# dcbtool sc eth3 dcb on
# dcbtool sc eth3 pfc e:1
# dcbtool sc eth3 app:fcoe e:1
As a convenience there is a script that will confirm if DCB has been
configured correctly for FCoE. The script is run as follows,
<fcoe-utils source>/debug/dcbcheck.sh eth3
(note: this is on the root device, not the VLAN)
Follow the suggestions and repeatedly run the script until it states that
DCB is configured correctly.
3) Start fcoe
# service fcoe start
After a few moments your storage should appear (assuming everything is
configured correctly on the fabric)
4) Setup lldpad and fcoe to start when booting
# chkconfig lldpad on
# chkconfig fcoe on
##
# Reporting Errors
##################
Most steps cannot be skipped. Resolve issues before moving forward.
Please run '<fcoe-utils-src>/debug/fcoedump.sh eth3.<VID>-fcoe &>dump.out'
and provide the dump.out file with the bug report. Defects can be filed
against the distribution used, reported to the Ethernet vendor of the
card in use, or mailed to fcoe-devel@open-fcoe.org.
##
# Debugging Tools
###################
1) Discover VLANs manually
# fipvlan -a
Fibre Channel Forwarders Discovered
interface | VLAN | FCF MAC
------------------------------------
eth3 | 101 | 00:0d:ec:a3:3a:80
VLANs should be automatically discovered by the FCoE start up process which
adds a VLAN to the real device. The VLAN can be identified by the name
ethx.vid-fcoe.
2) fcping can be used to ping targets to verify connectivity.
# fcping -h eth3.<VID>-fcoe -F E10AEF -c 3
sending echo to 0xE10AEF
echo 1 REJECT received 0.116 ms
echo 2 REJECT received 0.222 ms
echo 3 REJECT received 0.226 ms
3 frames sent, 3 received 3 errors, 0.000% loss, avg. rt time 0.188 ms
