Fibre Channel (FC)
Fibre channel (FC) is a high-speed network communications protocol used to implement SANs.
- British spelling is used to distinguish from fiber optic cabling
- defined in the T11 ANSI standard
- A SAN based on a Fibre Channel (FC) Switched Fabric (FC-SW) involves three types of components:
- Initiator
- client device of the SAN
- e.g., file or database server installed with a fibre channel host bus adapter (HBA)
- client device of the SAN
- Target
- network port for a storage device
- e.g., single drives, RAID drive arrays, tape drives, and tape libraries
- Space on the storage devices is divided into logical volumes
- each identified by a 64-bit logical unit number (LUN)
- initiator operates storage devices with commands for:
- SCSI
- Serial Attach SCSI (SAS)
- SATA
- or Nonvolatile Memory Express (NVMe)
- most devices have multiple ports
- for load balancing and fault tolerance
- Fibre channel switch (FC switch) provides the interconnections between initiators and targets (a fabric)
- aka director
- switch topology and interconnections are designed to provide multiple paths between initiators and targets
- for fault tolerance and load balancing
- High performance FC switches are often referred to as directors
- Initiator
- initiators and targets are identified by 64-bit WorldWide Names (WWN)
- similar to network adapter MAC addresses
- Collectively, initiators and targets are referred to as nodes
- can be allocated their own WWN
- referred to as a WWNN (WorldWide Node Name)
- each port on a node can have its own WorldWide Port Name (WWPN)
- can be allocated their own WWN
- can use rates from 1GFC (1 Gbps) up to 128GFC
- uses dedicated:
- SFP+ and QSFP optical
- and twinax modular transceivers
- fibre Channel transceivers and Ethernet transceivers are not interchangeable
Info
- NVMe specification is typically preferred over the older SCSI interface
- depend on the use of fast, highly parallel SSDs
- Using NVMe in a networked environment is referred to as NVMe over Fabrics (NVMe-oF)
- NVMe over FC can be referred to as either FC-NVMe or NVMe/FC
Converged Ethernet
- why use Fibre Channel fabric over Ethernet?
- a SAN requires quality of service (QoS) mechanisms to ensure flow control and guaranteed delivery
- new iteration of Ethernet has been developed as an alternative SAN fabric
- referred to as lossless Ethernet, Data Center Ethernet, or Converged Enhanced Ethernet
Fibre Channel over Ethernet (FCoE) is a means of delivering Fibre Channel packets over lossless Ethernet components.
- requires special 10/40/100G adapters
- combine the function of NIC and HBA
- referred to as converged network adapters (CNAs)
NVMe over Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE)
is a way of offloading storage transfers from the CPU and OS to improve performance, compared to NVMe/FC.
- more modern option than FCoE
TCP/IP
Internet Small Computer Systems Interface (iSCSI) is an IP tunneling protocol that enables the transfer of SCSI data over an IP-based network.
- works with ordinary Ethernet network adapters and switches
- can be used to link SANs
- also seen as an alternative to Fibre Channel or Converged Ethernet
- works with regular Ethernet adapters and switches
- also seen as an alternative to Fibre Channel or Converged Ethernet
NVMe over TCP (NVMe/TCP) uses the reliability mechanisms built into TCP to substitute for the lossless mechanisms of FC or CE.
- another option for SANs
- greater packet header and latency compared to NVMe/FC or RoCE
- but use of standard Ethernet products can simplify procurement and support procedures
Info
- A SAN should not be implemented on the same cabling as a production data network
- even if technologies such as iSCSI and NVMe/TCP make that technically possible
- performance of the SAN will be heavily impacted
- As a best practice,
- implement a dedicated network infrastructure that is restricted to only SAN traffic