view Side-By-Side changes
IP and ARP Over FC Working Group Murali Rajagopal INTERNET-DRAFT Raj Bhagwat<draft-ietf-ipfc-fibre-channel-00.txt><draft-ietf-ipfc-fibre-channel-01.txt> Wayne Rickard (ExpiresDec 22, 1998)Mar 18, 1999) (Gadzoox Networks) IP and ARP over Fibre Channel Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as Reference material or to cite them other than as ``work in progress''. To view the entire list of currentInternet-Drafts,Internet-Draft, please checkthe "1id-abstracts.txt"the``1id-abstracts.txt'' listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa),ftp.nordu.net (Northernnic.nordu.net ( Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). Abstract Fibre Channel (FC) is a high speed serial interface technology that supports several higher layer protocols including Small Computer System Interface (SCSI) and Internet Protocol(IP). Until now, SCSI has been the only widely used protocol overFibre Channel.FC. ExistingFibre ChannelFC standards [3] do not adequately specify how IP packets may be transported overFibre ChannelFC and how IP addresses are resolved to FC addresses. The purpose of this document is to specify a way of encapsulating IP and Address Resolution Protocol(ARP) over Fibre Channel and also to describe amechanismmechanism(s) for IP address resolution. 1. IntroductionFibre ChannelFC is a gigabit speed networking technology primarily used for Storage Area Networking (SAN). FC is standardized under American National Standards Institute (ANSI)and has specified a number of documents describing its protocols, operations, and services. Need: Currently, Fibre Channel is predominantly used for communication between storage devices and servers using the SCSI protocol, with most of the servers still communicating with each other over LANs. Although, the Fibre Channel standard [3] has architecturally defined support for IP encapsulation and address resolution, it is inadequately specified. ([3] prohibits broadcasts thus loops are not covered; [10] has no support for Class 3) Rajagopal, Bhagawat, Rickard [Page 1] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998covered; [10] has no support for Class 3)This has lead to a nonstandard way of using IP over FC in the past. Once, such a standard method is completely specified, servers can directly communicate with each other using IP over FC, possibly boosting performance in Server host-to-host communications. This technique will be especially useful in a Clustering Application. Objective: The major objective of this specification is to promote inter- operable implementations of IP over Fibre Channel. This specification describes a method for encapsulating IPv4 and Address ResolutionprotocolProtocol (ARP) packets over Fibre Channel. This specification accommodates any FC topology (loop, fabric, or point- to-point) and any FC class of service (1, 2 or 3). Use of IEEE 802.2 LLC/SNAP encapsulation for IP and ARP as specified in this document shall not preclude the use of same encapsulation technique for other protocol stacks (e.g. IPX, AppleTalk). Organization: Section 2 states the problem that is solved in this specification. Section 3 describes the techniques used for encapsulating IP and ARP packets in a FC sequence. Section 4 discusses ARP (IP address to MAC address) and the required mappings and operation. Section 5 discusses the FC Layer mappings (MAC address to Port_ID). Section 6 provides a discussion on validation of the FC-layer mapping for the different FC topologies. Section 7 describes the "Exchange" Management in FC. Section 8 is a summary section and provides a quick summary of the FC header settings, FC Link Service Commands, and a summarized reference to features supported in ARP, FC Sequences, FC Exchanges, and FC Login Parameters. Appendix A provides a brief overview of the FC Protocols and Networks along with a list of acronyms and a glossary of FC Terms used in this specification. Appendix B addresses reliability in Class 3. 2. Problem Statement This draft addresses two problems: - A sequence format definition and encapsulation mechanism for IP and ARP packets over FC -AnA mechanism(s) Address Resolutionmechanism.As noted earlier, the existing FC Standards [3], [10] are inadequate. A solution to both problems has been proposed by the Fibre Channel Association (FCA)[1]. FCA is a industry consortium of Fibre Channel vendor companies and not a standards body. This draft specification is largely based on the proposed solution in [1] and is an attempt to provide a standardized specification addressing both the above stated problems. 3. IP/ARP Encapsulation 3.1 FC Frame Format Rajagopal, Bhagawat, Rickard [Page 2] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 19983. IP/ARP Encapsulation 3.1 FC Frame FormatAll FC frames have a standard format much like LAN 802.x protocols. (See Appendix A for Fibre Channel related Acronyms and Glossary of Terms.) However, the exact size of each frame varies depending on the sizes of the variable fields. The FC frame structure is shown in Fig. 1. +-------+--------+-----------+----//-------+-----+-----+ | SOF |Frame |Optional | Payload |CRC | EOF | | (4B) |Header |Header | |(4B) |(4B) | |||(24B) |<----------------------->| | | ||(24B)| | (0-2112B) | | | +-------+--------+-----------+----//-------+-----+-----+ Fig. 1 FC Frame Format The Start of Frame (SOF) and End of Frame (EOF) are both 4 bytes long and act as frame delimiters. The CRC is 4 bytes long and uses the same 32-bit polynomial used in FDDI and is specified in ANSI X3.139 Fiber Distributed Data Interface. The Frame Header is 24 bytes long and has several fields associated with identification and control of the payload. The values and options for the fields that are relevant to the IP and ARP payloads will be discussed later. A FC Optional Header allows up to 4 optional header fields: - An Expiration Security Header (16 bytes) - Network (16 bytes) - Association (32 bytes) - Device (up to 64 bytes). The IP and ARP FC sequences are required to carry the Network_Header optional header field which is 16 bytes long. Other types of optional headers are prohibited. The use of the Network_Header for the IP and ARP payload encapsulation is described below. InFCFC, an application level payload is called a Sequence. Typically, a Sequence consists of more than one frame. Larger user data is segmented and reassembled using two methods: Sequence Count and Relative Offset. Use of Sequence Count is straight forward and data blocks are sent using frames with increasing sequence counts (modulo 16). With Relative Offset, frames could temporally arrive out of order. When IP and ARP form the FC payload then only the First Frame of the logical Sequence shall include the FC Network_Header.Care(Care should exercised when this is the case. Note that the physical temporal ordering of the frames can be different as a result of travesing through a "Fabric".) Fig. 2 shows the logical First Frame and logical subsequent frames Rajagopal, Bhagawat, Rickard [Page 3] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998and logical subsequent frames rather than temporal ordering.First Frame of a Logical FC Sequence ---+------------+---------------------------+----------//----------+--- | FC Header | FC Network Header | FC Sequence Data | ---+------------+---------------------------+---------//-----------+--- Subsequent Frames of a Logical FC Sequence --+-----------+----------//------+-- | FC Header | FC Sequence Data | --+-----------+----------//------+-- Fig. 2 FC Network Header in a Frame Sequence The SOF, CRC, EOF control fields and other optional headers have been omitted in the figure for clarity. 3.2 MTU The Maximum Transmission Unit (MTU) for IP is defined as the length of the IP packet, including IP headers. The theoretical maximum size of an IP Packet is 65,535 bytes. In FC-4 the transmission unit is a "Information Unit" and not frames. An N_Port may transmit an Information Unit using multiple frames. The receiving N_Port will assemble the frames to reconstruct the sent Information Unit. The size of a single Information Unit is limited to 2^32-1, which is very large. However, restricting the IP over FC MTU helps in buffer resource allocation at N_Ports. A MTU of 65,280 bytes allows for up to 256 bytes of overhead. The IEEE 802.2 LLC/SNAP headers requires 8 bytes, leaving the rest 248 bytes for future uses. There shall be a one-to-one mapping between an IP packet and a FC sequence. In other words, one IP packet shall always map to only one FC Sequence. Note that, although the FC physical frame MTU is limited to 2112 bytes, it is hidden from IP and does not affect the IP MTU at FC-4. 3.3 FC Port and Node Network Addresses FC devices are identified by Nodes and Ports. A Node is a collection of one or more Ports identified by a unique nonvolatile(unchangeable)64-bit World Wide Node name(WWN_N).(WW_NN). Each Port in a node, is identified with a unique nonvolatile 64-bit World Wide Port name(WWP_N),(WW_PN), and a volatile(changeable)Port_ID. Port_ID are 24-bits. In a FC frame header, the Port_ID is referred to as S_ID (Source ID) to identify the port originating a frame, and D_ID to identify the destination port. The Port_ID of a given port isvolatile (changeable).volatile. (Themechanisms throughmechanism(s) by which a Port_ID may change in a FC topologyareis outside the scope of this document.) FC specifies a Network Address Authority (NAA) to distinguish between the various name registration authorities that may be used toRajagopal, Bhagawat, Rickard [Page 4] Internet-Draft IP and ARP over Fibre Channel June 22 1998identify theWWP_NWW_PN and theWWP_N.WW_NN. A 4-bit NAA identifier, 12-bit field set to 0x000 and an IEEE 48-bit MAC address together make the Rajagopal, Bhagawat, Rickard [Page 4] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 64-bitWWN_NWW_NN or theWWP_NWW_PN addresses [2]. In a single port Node, theWWN_NWW_NN and theWWP_NWW_PN may be identical. TheWWN_PWW_PN names of the source and destinations are carried in the FC Network Header. The format of the FC Network Header is shown in Fig. 3 and defined in the FC standards [2]. The Network header is normally optional in FC but mandatory in this specification. The 4 most significant bits in each address denotes the Network Address Authority (NAA) type. In this specification, the source and destination NAA binary pattern '0001' indicates the IEEE-48 bit MAC address and is the only code point that is allowed.TheThis NAA field value allows FC networks to be bridged with other FC networks or traditional LANs. The Source (Destination) MAC address occupies the lower 48 bits of the Network_Source_Address (Network_Dest_Address), and the upper 12 bits are set to 0x000. +--------+---------------------------------------+ | D_NAA |Network_Dest_Address (High-order bits) | |(4 bits)| (28 bits) | +--------+---------------------------------------+ | Network_Dest_Address (Low-order bits) | | (32 bits) | +--------+---------------------------------------+ | S_NAA |Network_Source_Address(High-order bits)| |(4 bits)| (28 bits) | +--------+---------------------------------------+ | Network_Source_Address (Low-order bit) | | (32 bits) | +--------+---------------------------------------+ Fig. 3 Format of the Network Header Field 3.4 FC Payload Format The payload of an FC sequence carrying an IP packet shall use the format shown in Fig. 4. Fig. 5 shows the format when the payload is an ARP packet. However, both formats use the 8-byte LLC/SNAP header. +-----------------+-----//----------+-------------//------------+ | LLC/SNAP Header | IP Header | IP Data | | (8 bytes) | (20 bytes min.) | (65280 -IP Header) bytes | +-----------------+-----//----------+-------------//------------+ Fig. 4 Format of FC Sequence Payload carrying IP +-----------------+-------------------+ | LLC/SNAP Header | ARP Packet | | (8 bytes) | (28 bytes) | +-----------------+-------------------+ Fig. 5 Format of FC Sequence Payload carrying ARPRajagopal, Bhagawat, Rickard [Page 5] Internet-Draft IP and ARP over Fibre Channel June 22 1998As notedabove,earlier, since FC frames belonging to the same Sequence can be delivered out of order over a Fabric, the IP Header must appear in Rajagopal, Bhagawat, Rickard [Page 5] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 the frame that has relativerandomoffset of 0. A Logical Link Control (LLC) field along with a Sub Network Access Protocol (SNAP) field is a method used to identify routed and bridged non-OSI protocol PDUs and is defined in IEEE 802.2 and applied to IP in [8]. In LLC Type 1 operation (i.e., unacknowledged connectionless mode), the LLC header is 3-bytes long and consists of a 1-byte Destination Service Access Point (DSAP)field, a 1-byte Source Service Access Point (SSAP)field, and a 1-byte Control field as shown in Fig. 6. +----------+----------+----------+ | DSAP | SSAP | CTRL | | (1 byte) | (1 byte | (1 byte) | +----------+----------+----------+ Fig. 6 LLC Format The LLC's DSAP and SSAP values of 0xAA indicate that a IEEE 802.2 SNAP header follows. The LLC's CTRL value equal to 0x03 specifies Unnumbered Information Command PDU. The LLC header value shall be 0xAA-AA-03. Other values of DSAP/SSAP indicate support for other protcols but are prohibited in this specification. The SNAP header is 5 bytes long and consists of a 3-byte Organizationally Unique Identifier (OUI) field and a 2-byte Protocol Identifier as shown in Fig. 7 +------+------+-------+------+------+ | OUI | PID | | ( 3 bytes) | (2 bytes) | +------+------+-------+------+------+ Fig. 7 SNAP Format SNAP was invented to "encapsulate" LAN frames within the payload. The SNAP OUI value 0x00-00-00 specifies that the PID is an EtherType (i.e., routed non-OSI protocol).The SNAP PID Type field specifiesAn OUI value of 0x00-80-C2 indicates Bridged Protocols. When theEtherType value. In particular,OUI value equals 0x00-00-00, the SNAP PID value of 0x08-00 indicates IP and a SNAP PID value of 0x08-06 indicates ARP. The complete LLC/SNAP header is shown in Fig. 8. +----------+----------+----------+-------+-------+-------+-------+------+ | DSAP | SSAP | CTRL | OUI | PID | | (1 byte) | (1 byte) | (1 byte) | ( 3 bytes) | (2 bytes | +----------+----------+----------+-------+-------+-------+-------+------+ Fig. 8 LLC/SNAP Header 3.5 ARP Packet Format The format of the encapsulated ARP packet is based on [9] and isshown in Fig. 9.Rajagopal, Bhagawat, Rickard [Page 6] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998 shown in Fig. 9. The 'HW Type' field shall be set to 0x00-01 Note: Technically, the correct HW Type value should be set to 0x00-06 according to RFC 1700 indicating IEEE 802 networks.The 'Protocol'However, as a practical matter a HW Type value of 0x00-06 is known to cause rejections from some Ethernet end stations when FC is bridged to Ethernet. Translational bridges are normally expected to change this fieldshall be setfrom Type 6 to0x08-001 and vice versa under these configurations but many do not. It is because of this reason the Type Code is set to 1 rather than 6. However, both HW Type values of 0x00-01 and 0x00-06 shall be accepted. The 'Protocol' field shall be set to 0x08-00 indicating IP protocol. The 'HW Addr Length' field shall be set to 0x06 indicating 6 bytes of HW address. The 'Protocol Addr Length' field shall be set to 0x04 indicating 4 bytes of IP address. The 'Operation' Code field shall be either 0x00-01 for Request or 0x00- 02 for Reply. +-------------------------+ | HW Type | 2 bytes +-------------------------+ | Protocol | 2 bytes +-------------------------+ | HW Addr Length | 1 byte +-------------------------+ | Protocol Addr Length | 1 byte +-------------------------+ | Op Code | 2 bytes +-------------------------+ | HW Addr of Sender | 6 bytes +-------------------------+ | Protocol Addr of Sender | 4 bytes +-------------------------+ | HW Addr of Target | 6 bytes +-------------------------+ | Protocol Addr of Target | 4 bytes +-------------------------+ Total 28 bytes Fig. 9 ARP Packet Format The 'HW Addr of Sender' field shall be the 6 byte IEEE MAC address of the sender. The 'Protocol Addr of Sender' field shall be the 4 byte IP address of the sender. Rajagopal, Bhagawat, Rickard [Page 7] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 The 'HW Addr of Target' field shall be set to zero if the 'Operation Code' field is set to 1. Otherwise, it shall be set to the 6 byte IEEE MAC address of the original sender of the ARP request. The 'Protocol Addr of Target' field shall be set to the 4 byte IP address of the target. The ARP packet is 28 bytes long in this particular application. The difference between an ARP Request Packet and an ARP Reply Packet is given below: 1. ARP Request packet: 'Operation' Code field = 0x00-01 and the 'HWRajagopal, Bhagawat, Rickard [Page 7] Internet-Draft IP and ARP over Fibre Channel June 22 1998Addr of Target' is set to 0x00-00-00-00-00-00. 2. ARP Reply packet: 'Operation' Code field = 0x00-02 and the 'HW Addr of Target' is appropriately set to theaddressextracted 'HW Addr of Sender' field from the ARP Requestpacket's Sender addresspacket; similarly, the 'Protocol Addr of Target' is set to the extracted 'Protocol Addr of Sender' field from the ARP Request packet An ARP Request message is defined as a FC broadcast sequence carrying the ARP Request packet. The exact mechanism used to broadcast a FC sequence depends on the topology and will be discussed in the next section. Compliant ARP broadcast messages shall include Network Headers. An ARP Reply message is defined as an ARP Reply packet encapsulated in a FC sequence. Compliant ARP Reply messages shall include Network Headers. 4. Address Resolution 4.1 Problem Description Address Resolution is concerned with associating IP addresses with FC Port addresses. As described earlier, FC device ports have two addresses: - a non-volatile unique 64-bit address called World Wide Port_Name(WWP_N)(WW_PN) - a volatile 24-bit address called a Port_ID (see Appendix A for a definition of Port_ID) The Address Resolution mechanism therefore will need two levels of mapping: 1. A mapping from IP address to theWWP_NWW_PN address(i.e., IEEE 48-bit MAC address) 2. A mapping fromWWP_NWW_PN to the Port_ID The address resolution problem is compounded by the fact that the Port_ID is volatile and the second mapping has to be validated before use. Moreover, this validation process can be different depending on Rajagopal, Bhagawat, Rickard [Page 8] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 the FC network topology used. Architecturally, the first level of mapping and control operation is handled by the ARP layer, and the second level of mapping and control by the FC layer. 4.2 ARP Layer Mapping and Operation Whenever a source FC port with a designated IP address wishes to send IP data to a destination FC port also with a designated IP address then, the following steps are taken: 1. The source port shall consult its local mapping tables to determine the <destination IP address, destinationWWP_N>. (Note, WWP_NWW_PN>. (Since the NAA= b'0001' the WW_PN address and 48-bit MAC addresswillconceptually mean the same thing in this discussion.)Rajagopal, Bhagawat, Rickard [Page 8] Internet-Draft IP and ARP over Fibre Channel June 22 19982. If such a mapping is found, then the source shall send the IP data to the port whoseWWP_NWW_PN address was found in the table.The corresponding destination Port_ID shall first be validated.3. If such a mapping is not found, then the source shall send an ARP broadcast message to its connected FC network in anticipation of getting a reply from the correct destination along with itsWWP_NWW_PN address. 4. When an ARP broadcast message is received by the destination it shall generate an ARP response. Since the ARP response must be addressed to a specific destination Port_ID, the FC layer mapping between the MAC address and Port_ID (of the ARP Request orginator) must be valid before the reply is sent. 4.2.1 ARP Broadcast in a Point-to-Point TopologyThere is no requirement forThe ARPsince the WWP_NRequest (Broadcast) and Reply mechansim described in Section 3.5 and 4.2 still applies, although there isknown after the two N_Ports carry out a N_Port Login,only one node thatis a PLOGI (See Appendix A).receives this. 4.2.2 ARP Broadcast in a Private Loop Topology In a private loop, the ARP broadcast message is sent using the broadcast method specified in the FC-AL [7]standard. 1. The source port shall first send an Open Broadcast Replicate primitive (OPN(fr))Signal forcing all the ports in the loop (except itself), to replicate the frames that they receive while examining the frame header's Destination_ID field. 2. The source port shall remove this OPN(fr) signal when it returns to it. 3. The loop is now ready to receive the ARP broadcast message and is sent as a broadcast sequence, that is using FC frames. Rajagopal, Bhagawat, Rickard [Page 9] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 4. The source shall now send a FC frame containing the ARP Request (ARP broadcast message), as a sequence in a Class 3 frame with the following FC Header D_ID field and F_CTL bits in the FC header set to: Destination ID<Word 0, bit 0:23>: D_ID = 0xFF-FF-FF Sequence Initiative <Word 2, bit23>: SI=0 Last Sequence <Word 2, bit 20>: LS=1 End Sequence <Word 2, bit 19>: ES=1. The above FCTL settings apply to single-frame broadcasts, as used in ARP sequences. This information is provided toRajagopal, Bhagawat, Rickard [Page 9] Internet-Draft IP and ARP over Fibre Channel June 22 1998clarify ARP Broadcast usage only, and should not be interpreted as prohibiting the use of multiframe sequence broadcastsby this specification.for other applications. 5. Compliant ARP broadcast sequences shall include Network Headers with destination MAC address in the Network Header set to 0xFF-FF-FF-FF-FF-FF and NAA = b'0001' 6. The destination port recognizing its IP address in the ARP packet shall respond with an ARP Reply message. 4.2.3 ARP Broadcast in a Public Loop Topology The following steps will be followed when a port is configured in a public loop: 1. A public loop device attached to a fabric through an FL_Port shall not use the OPN(fr) signal primitive. Rather, it shall send the broadcast sequence to the FL_Port at AL_PA = 0x00. 2. A fabric shall propagate the broadcast to all other ports including the FL_Port which the broadcast arrived on. This includes all F_Ports, and other FL_Ports. 3. On each FL_Port, the fabric shall first propagate the broadcast by first using the primitive signal OPNfr, in order to prepare the loop to receive the broadcastsequencesequence. 4. A broadcast sequence is now sent on all ports (all FL_ports, F_Ports)in Class 3 frame with: Destination ID <Word 0, bit 23:0>: D_ID = 0xFF-FF-FF Sequence Initiative <Word 2, bit23>: SI=0 Last Sequence <Word 2, bit 20>: LS=1 End Sequence <Word 2, bit 19>: ES=1. 5. Compliant ARP broadcast sequences shall include Network Headers Rajagopal, Bhagawat, Rickard [Page 10] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 with destination MAC address in the Network Header set to 0xFF-FF-FF-FF-FF-FF and NAA = b'0001' 6. The destination port recognizing its IP address in the ARP packet shall respond with an ARP Reply message. 4.2.4 ARP Operation in a Fabric Topology 1. Nodes directly attached to fabric do not require the OPN(fr) primitive signal. 2. A broadcast sequence is now sent on all ports (all FL_ports, F_Ports)in Class 3 frame with: Destination ID <Word 0, bit 23:0>: D_ID = 0xFF-FF-FFRajagopal, Bhagawat, Rickard [Page 10] Internet-Draft IP and ARP over Fibre Channel June 22 1998Sequence Initiative <Word 2, bit23>: SI=0 Last Sequence <Word 2, bit 20>: LS=1 End Sequence <Word 2, bit 19>: ES=1. 3. Compliant ARP broadcast sequences shall include Network Headers with destination MAC address in the Network Header set to 0xFF-FF-FF-FF-FF-FF and NAA = b'0001' 4. The destination port recognizing its IP address in the ARP packet shall respond with an ARP Reply5.05. Mechanisms for Maintaining FC Layer Mappings FC layer mapping between theMAC addressWW_PN and the Port_ID is independent of the ARP mechanism and is more closely associated with the details of the FC protocols.The section presents several possibleTwo main mechanisms - Name Server and FARP that may be usedfor maintaining FC-layer mappings, that is,to create and maintainMAC AddressWW_PN toPort Address tables.Port_ID tables are described here. Other less formal mechanisms are described in Appendix C. The preferred method is a configuration and administration issue, and may be implementation-dependent.Each5.1 Use of Name Server This methodshould have some mechanism to ensure PLOGI has completed successfully before dataissent. A related concernused inlarge networksenvironments where a Name Server islimiting concurrent loginsavailable[4]. 1. A Name Server may be referenced toonly those ports with active IP traffic. 5.1 Login on Cached Mapping Information This method insulatesresolve unmapped WW_PN addresses. 2. Any upper layer send request for which there is not a Port_ID to WW_PN mapping can trigger a query to a name server. The WW_PN must be re-formatted in thelevel performing LOGIN from64-bit WW_PN format before thelevel interpreting ARP. Itquery ismore accommodatingissued. 3. The format ofnon-ARP mechanisms for buildingtheFC-layer mapping table. 1. Broadcast messages that carryName Server query and response is outside the scope of this document. See[4] for aNetwork Headertypical example and Rajagopal, Bhagawat, Rickard [Page 11] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 [14] for a Name Server implementation. 4. The query response from the Name Server must contain theS_ID onPort_ID associated with theFC-header and WWP_NWW_PN specified in theNetwork-header. Cachingquery. 5. Normal Port Login procedures follow at thisinformation providespoint before packets can be forwarded to acorrelation of Port_IDport. 5.2 FARP The Fibre Channel Address Resolution Protocol (FARP) is a method using ELS commands toWWP_N. Ifresolve <WW_PN, D_ID> mapping in environments without a Name Server. That is, when thereceived Broadcast messageWW_PN iscompliant with this specification, the WWP_N will beknown, but not theMAC Address.D_ID and a Name Server service doesn't exist. Thismethod may also accommodate other NAA types. 2. The WWP_N is "available" ifsituation arises, for instance, when Loginhas been performedtables entries expire. The FARP Extended Link Service Broadcast Request (FARP-REQ) shall resolve Port_IDs of communicating Fibre Channel devices. A FARP-REQ can be used totheretrieve a specific N_Port's current Port_ID given the unique WW_PN andflagged. IfWW_NN. This is accomplished by requesting either a FARP Response ELS Unicast command (FARP-RES), or by indicating that the Responder N_Port shall perform a loginhas not been performed,with theWWP_N is "unavailable". 3. If an outbound packetFARP Requester. No sequence initiatives are transferred with the FARP-REQ and the FARP-RES and therefore no Reply (ACCEPT or REJECT) isdestinedsent for either message. The FARP-RES is aport thatmessage response sent by a higher level entity at the Responder host. Protocol: FARP Request Sequence : FARP-REQ (ELS broadcast) No Reply Sequence FARP Response Sequence : FARP-RES (ELS command) No Reply Sequence Format: FT-1 Addressing: - For a FARP-REQ, The S_ID designates the Requester N_Port requesting addressing information. The D_ID is"unavailable",thecached informationbroadcast identifier, 0xFF-FF-FF. - For a FARP-RES, the S_ID designates the N_Port ID of the device matching the Responder Address Information in the FARP Request. The D_ID isused to look upthePort_ID. 4. After sending an ELS PLOGI command (Port Login) toN_Port ID of thePort, wait. By waiting for an outbound packet before initiating login, login resourcesdevice that initiated the FARP request. Payload: There arereserved only for those ports which2 formats of the FARP-REQ payload depending on the address information carried. Rajagopal, Bhagawat, Rickard [Page11]12] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998wish to establish communication. 5. After Port Login completes (ACC received), the outbound packet can be forwarded. At this point in time, both ends have the necessary information+-----------------------------------------------------------------+ | FARP-REQ Payload | +-----------------------------+---------+-------------------------+ | Field | Size | | | |(Bytes) | Remarks | +-----------------------------+---------+-------------------------+ | 0x54-00-00-00 | 4 | | +-----------------------------+---------+-------------------------+ | Match Address Code Points | 1 | | +-----------------------------+---------+-------------------------+ | Port_ID of Requester | 3 | | +-----------------------------+---------+-------------------------+ | Responder Flags | 1 | | +-----------------------------+---------+-------------------------+ | Port_ID of Responder | 3 | set tocomplete their <IP address, MAC Address, Port_ID> association. 5.2 Login on ARP Parsing This method performs LOGIN sooner by parsing ARP before passing it up0x00-00-00 | +-----------------------------+---------+-------------------------+ |WW_PN of Requester (FARP-REQ)| 8 | | +-----------------------------+---------+-------------------------+ |WW_NN of Requester (FARP-REQ)| 8 | | +-----------------------------+---------+-------------------------+ |WW_PN of Responder | 8 | | +-----------------------------+---------+-------------------------+ |WW_NN of Responder | 8 | | +-----------------------------+---------+-------------------------+ +-----------------------------------------------------------------+ | FARP-REQ Payload | +-----------------------------+---------+-------------------------+ | Field | Size | | | |(Bytes) | Remarks | +-----------------------------+---------+-------------------------+ | 0x54-00-00-00 | 4 | | +-----------------------------+---------+-------------------------+ | Match Address Code Points | 1 | | +-----------------------------+---------+-------------------------+ | Port_ID of Requester | 3 | | +-----------------------------+---------+-------------------------+ | Responder Flags | 1 | | +-----------------------------+---------+-------------------------+ | Port_ID of Responder | 3 | set tohigher levels for IP/MAC0x00-00-00 | +-----------------------------+---------+-------------------------+ |IP Addresscorrelation. It requires a low- level awarenessofthe IP address,Requester | 16 |IPv4 Add.=lower 32 bits | | (FARP-REQ) | |IPv6 Add. = 128 bits | +-----------------------------+---------+-------------------------+ |IP Address of Responder | 16 |IPv4 Add. = lower 32 bits| | | |IPv6 Add. = 128 bits | +-----------------------------+---------+-------------------------+ Both formats carry common fields: command code, Match Address Code Point, Port_ID of Requester, Responder Flags, andis therefore protocol- specific. 1. When an ARP Broadcast Message is received, extract the S_ID fromPort_ID of Responder. The first fromat carries theFC headerWW_PN and WW_NN of both thecorresponding Network_Source_Address from the Network Header. 2. ParseRequester and Responder while theARP payload to determine if (a) you aresecond format carries thetarget of the ARP request (TargetIPAddress match),addresses of Rajagopal, Bhagawat, Rickard [Page 13] Internet-Draft IP and(b) you are currently logged in with the port (Port_ID = S_ID) originating theARPbroadcast. 3. Passover Fibre Channel Sep 18 1998 Requester and Responder. Note that theARP to higher level for ARP Response generation. 4. If a Port Login is required, an ELS PLOGI command (Port Login)NAA issent immediatelyimplicitly assumed tothebe defined to be equal to b'0001' indicating IEEE-48-bit MAC addresses are contained in Worl Wide Portoriginatingand Node Names. In theARP Broadcast. 5. After Port Login completes, an ARP response canfirst format the "WW_PN of Responder" and "WW_NN of Responder" fields should beforwarded. Note that there are two possible scenarios: - The ACC to PLOGI returns beforefilled in with theARP reply is processedNode and Port Names of theARP Reply is immediately forwarded. -desired Responder. TheARP replyMatch Address Code Points define what addresses to match based on these code points. The Responder Flags define what Responder action if the result of the Match Address Code Points isdelayed, waiting for ACC (successful Login). 6. At this point in time, both ends havesuccessful. In thenecessary information to complete their <IP address, MAC Address, Port_ID> association. 5.3 Usefirst format, the FARP-REQ Requester supplies the WW_PN ofName Server This methodthe Responder, the WW_NN of the Responder, or both. WW_PN in FARP ispreferredthe 8-byte WW_PN of the Requester/ Responder to the FARP-REQ. WW_NN inenvironments where a Name ServerFARP isrequired [4]. Compliant topologies require a Name Server, while [5] devices may not be ablethe 8-byte WW_NN of the Requester/ Responder toaccessthewell-known Name Server address, even if one exists. 1. A Name Server may be referenced to resolve unmapped MAC addresses. 2. Any upper layer send request for which there is not a Port_ID to MAC address mapping can triggerFARP-REQ. In the second format, the FARP-REQ Requester supplies either aquery to32-bit IPv4 Address or aname server. Rajagopal, Bhagawat, Rickard [Page 12] Internet-Draft IP and ARP over Fibre Channel June 22 1998 3. The format128-bit IPv6 address of theName Server queryRequester andresponse is outside the scopeResponder. Port_ID ofthis document. See FC-FLA [4] for a typical example. 4. A preferred Name Server implementationRequester: isdescribed in [ns008.pdf on ftp.network.com]. The MAC address must be re-formattedthe 24-bit Port_ID used in the64-bit WWP_N format beforeS_ID field of thequeryFARP-REQ header. Port_ID of Responder: isissued. 5. The query response from the Name Server must containthe 24-bit Port_IDassociated with the MAC Address specifiedused in thequery. 6. SendS_ID field of the FARP-RES header. Responder Flags: is anELS PLOGI command (Port Login) to8-bit field (bits 0-7) that defines thePort. 7. After Port Login completes,action of theoutbound packet can be forwarded. 8. At this pointResponder. This field is only valid intime, both endsa FARP-REQ. FARP-REQ is an ELS broadcast command. You do not havethe necessary information to complete their <IP address, MAC Address, Port_ID association>. 5.4 LogintoEveryone In Fibre Channel topologies with a limited number of ports, it maybeefficient to unconditionally login to each port. This method is discouragedlogged infabric and public loop environments. Afterto issue a FARP request. Possible Responder Actions: Port Logincompletes,(P_LOGI) Sent to theMAC AddressPort Identified by " Requester Port_ID" field when responder bit 0 (INIT_PLOGI) == binary '1' FARP Response Sequence (FARP-RES) Sent toPort_ID Address tables can be constructed. 5.5 Static Table In some loop environments with a limited number of ports, a static mapping from a MAC Address to Port_ID (D_ID or AL_PA) may be maintained. The FC layer will always know the destination Port_ID based on the table. The table is typically downloaded intothedriver at configuration time. This method scales poorly, and is therefore not recommended. 5.6 FARP The Fibre Channel Address Resolution Protocol (FARP) is a method using ELS commands to resolve <WWP_N, D_ID> mapping in environments without a Name Server. That is,Port Identified by "Requester Port_ID" field when responder bit 1 (INIT_FARP-RES) == binary '1' Bits 2 (INIT-PLOGI) = binary '1' and Bit 3 (INIT_FARP-RES) = binary '1' at theWWP_Nsame time isknown, but not the D_ID and a Name Server service doesn't exist. This situation arises, for instance, when Login tables entries expire. The FARP Extended Link Service Request shall resolve Port_IDsdisallowed. Recipients ofcommunicating Fibre Channel devices. A FARP Request can be used to retrieve a specific N_Port's current Port_ID giventheunique WWP_N and WWN_N. This is accomplished by requesting either aFARPResponseRequest ELScommand, or by indicating that the Responder N_Portshallperformnot issue alogin with theService Reject (LS_RJT) if FARPOriginator.is not supported. Rajagopal, Bhagawat, Rickard [Page13]14] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998Protocol: FARP Request Sequence (ELS broadcast) No Reply Sequence FARP Response Sequence (ELS command) Accept Reply Sequence Format: FT-1 Addressing: - For a FARP Request, The S_ID designates the Originator N_Port requesting addressing information. The D_ID is the broadcast identifier, 0xFF-FF-FF. - For a FARP Response,Table below indicates theS_ID designates the N_Port ID of the device matchingaction performed for each bit. If no bits are set, the ResponderAddress Information in thewill take no action. +----------+-------------------------------------------------------+ | | FARPRequest. The D_ID isResponder Flag | +----------+--------------+----------------------------------------+ | Bit | Bit Name | Action | | Position | | | +----------+--------------+----------------------------------------+ | 0 | INIT_PLOGI | Initiate P_LOGI to theN_Port IDRequester | +----------+--------------+----------------------------------------+ | 1 | INIT_FARP-RES| Send FARP_RES message to Requester | +----------+--------------+----------------------------------------+ | 2 to 7 | Reserved | | +----------+--------------+----------------------------------------+ For each recipient of thedevice that initiatedFARP-REQ Broadcast ELS, theFARP request. Payload: The formatrecipients match one or more addresses based on the encoded bits of theFARP Request payload"FARP Match Address Code Points" field. This isas follows: +-----------------------------------------+---------+shown in the following table: +------------------------------------------------------------------+ | FARPRequest PayloadMatch Address Code Points | +------------------------------------------------------------------+ |+-----------------------------------------+---------+LSBits |FieldBit name |SizeAction | +-----------+------------------+-----------------------------------+ ||(Bytes)0000 | Reserved |+-----------------------------------------+---------+|0x54-00-00-00+-----------+------------------+-----------------------------------+ |40001 |+-----------------------------------------+---------+MATCH_WW_PN | Match on WW_PN of ResponderFlags|1+-----------+------------------+-----------------------------------+ |+-----------------------------------------+---------+0010 |Port_IDMATCH_WW_NN | Match on WW_NN ofOriginatorResponder |3+-----------+------------------+-----------------------------------+ |+-----------------------------------------+---------+ |WWP_N of Originator0011 |8MATCH_WW_PN & | Match on both WW_PN and |+-----------------------------------------+---------+ |WWN_N of Originator|8|+-----------------------------------------+---------+ |WWP_NMATCH_WW_NN | WW_NN of Responder |8+-----------+------------------+-----------------------------------+ | 0100 to | Reserved | | | 1000 | | | +-----------+------------------+-----------------------------------+ |+-----------------------------------------+---------+ |WWN_N1001 | MATCH_IPv4 | Match on IPv4 Address ofResponderResponder| +-----------+------------------+-----------------------------------+ |81010 |+-----------------------------------------+---------+ The "WWP_NMATCH_IPv6 | Match on IPv6 Address ofResponder" and "WWN_NResponder| +-----------+------------------+-----------------------------------+ | 1011 to | Reserved | | | 1111 | | | +-----------+------------------+-----------------------------------+ Note that bit-3 ofResponder" fields should be filled in withtheNodeLSB differentiates between World_Wides addresses andPort Names of the desired Responder, while the Responder Flags define what actionIP addresses. If there no matches then a silent behavior from the Respondershould take. Theis valid. FARPRequest Originator can supplyResponse (FARP-RES) is an ELS command directed to theWWP_NPort_ID of theResponder,FARP-REQ Requester. You do not have to be logged in to theWWN_NFARP- REQ Requester to issue a FARP-RES. The format of theResponder, or both. Corresponding bitsFARP-RES payload is as follows: Rajagopal, Bhagawat, Rickard [Page14]15] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998in the "Responder Flags" field should also be set. WWP_N in FARP is the 8-byte WWP_N of the Originator / Responder to the FARP request. WWN_N in FARP is the 8-byte WWN_N of the Originator / Responder to the FARP request. Port_ID: is the 24-bit+---------------------------------------------------------------------+ | FARP-RES Payload with WW-Names | +-------------------------------------+---------+---------------------+ | Field | Size | Remarks | | | (Bytes) | | +-------------------------------------+---------+---------------------+ | 0x55-00-00-00 | 4 | | +-------------------------------------+---------+---------------------+ | Match Address Code Points | 1 | | +-------------------------------------+---------+---------------------+ | Port_IDused in the S_ID field of the FARP Request or FARP Response header. Responder Flags: is an 8-bit field (bits 0-7) that defines the actionofthe Responder. This field is only valid in a FARP Request. Table below indicates the action performed for each bit. If no bits are set, the Responder will take no action. +----------+-------------------------------------------------------+Requester | 3 | | +-------------------------------------+---------+---------------------+ |FARPResponderFlagFlags |+----------+--------------+----------------------------------------+1 |BitNot used |Bit Name+-------------------------------------+---------+---------------------+ |ActionPort_ID of Responder | 3 |Position| +-------------------------------------+---------+---------------------+ |WW_PN of Requester (FARP-REQ) | 8 |+----------+--------------+----------------------------------------+|0+-------------------------------------+---------+---------------------+ |WW_NN of Requester (FARP-REQ) |MATCH_PORT8 |Match on WWP_N| +-------------------------------------+---------+---------------------+ |WW_PN of Responder |+----------+--------------+----------------------------------------+ | 18 |MATCH_NODE|Match on WWN_N+-------------------------------------+---------+---------------------+ |WW_NN of Responder |+----------+--------------+----------------------------------------+8 |2|INIT_PLOGI+-------------------------------------+---------+---------------------+ +---------------------------------------------------------------------+ |Initiate P_LOGI to the OriginatorFARP-RES Payload with IP Addresses |+----------+--------------+----------------------------------------++-------------------------------------+---------+---------------------+ |3Field |INIT_FARPRSize |Send FARP Response ELS to Originator|+----------+--------------+----------------------------------------+| | (Bytes) | Remarks | +-------------------------------------+---------+---------------------+ | 0x54-00-00-00 | 4 |Reserved| +-------------------------------------+---------+---------------------+ |+----------+--------------+----------------------------------------+Match Address Code Points |51 |Reserved| +-------------------------------------+---------+---------------------+ |+----------+--------------+----------------------------------------+Port_ID of Requester |63 |Reserved| +-------------------------------------+---------+---------------------+ |+----------+--------------+----------------------------------------+Responder Flags |71 |ReservedNot used | +-------------------------------------+---------+---------------------+ |+----------+--------------+----------------------------------------+ FARP Request is an ELS broadcast command. You do not have to be logged in to issue a FARP request. PossiblePort_ID of ResponderActions: Port Login (P_LOGI) Sent to the Port Identified by " Originator Port_ID" field when responder bit 2 (INIT_PLOGI) == binary '1' FARP Response (FARP) Sequence Sent to| 3 | | +-------------------------------------+---------+---------------------+ |IP Address of Requester (FARP-REQ) | 16 |IPv4 Add.= lo 32 bits| | | |IPv6 Add.= 128 bits | +-------------------------------------+---------+---------------------+ |IP Address of Responder | 16 |IPv4 Add.= lo 32 bits| | | |IPv6 Add.= 128 bits | +-------------------------------------+---------+---------------------+ 6. FC layer Address Validation At all times, thePort Identified by "Originator Port_ID" field<MAC Address, Port_ID> mapping has to be validated before use. There are many events that can invalidate this mapping. The following discussion addresses conditions whenresponder bit 3 (INIT_FARPR) == binary '1'such a validation is required. Rajagopal, Bhagawat, Rickard [Page15]16] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998Recipients of the FARP Request ELS shall not issue6.1 General Discussion After aService Reject (LS_RJT) if FARP is not supported. For each recipient of the FARP Request Broadcast ELS, the recipients WWN_N and/or WWP_N is matched againstlink interruption occurs, the"WWN_N of Responder" and "WWP_NPort_ID ofResponder" fields based on the Responder Flags. If the MATCH_PORT bit is set, the Responder WWP_N is compared with the recipients WWP_N. Ifa port may change. After theMATCH_NODE bit is set,interruption, theResponder WWN_N is comparedPort_IDs of all other ports that have previously performed PLOGI (N_Port Login) withthe recipients WWN_N. If both bits are set, both are compared, and boththis port may haveto match. If no match is made, the sequence is ignoredchanged, andno action is taken. If thereits own Port_ID may have changed. Because of this, address validation is required after amatch, the "Responder Flags" field defines what action to take. This logic is shownLIP inthe following table: +-------------------+-------------------+-------------------+ | Compare | MATCH_PORT | MATCH_NODE | +-------------------+-------------------+-------------------+ | Ignore | 0 | 0 | +-------------------+-------------------+-------------------+ | Compare | | | | Responder WWP_N | 1 | 0 | | with | | | | Recipient WWP_N | | | +-------------------+-------------------+-------------------+ | Compare | | | | Responder WWN_N | 0 | 1 | | with | | | | Recipient WWN_N | | | +-------------------+-------------------+-------------------+ | Compare | | | | Responder WWP_N & | | | | WWN_N | 1 | 1 | | with | | | | Recipient WWP_N | | | | & WWN_N | | | +-------------------+-------------------+-------------------+ FARP Responsea loop topology [7]or after NOS/OLS in a point-to-point topology [6]. Port_IDs will not change as a result of Link Reset(LR),thus address validation is not required. In addition to actively validating devices after a link interruption, if a port receives any FC-4 data frames (other than broadcast frames), from a port that is not currently logged in, then it shall send anELSexplicit Extended Link Service (ELS) Request logout (LOGO) commanddirectedtothe Originator of the FARP Request. You do not havethat port. ELS commands (Requests and Replies) are used by an N_Port tobe logged insolicit a destination port (F_Port or N_Port) tothe FARPperform some link-level function or service.) The LOGO RequestOriginatoris used toissue a FARP Response. Reply Link Service Sequence: Service Reject (LS_RJT) Signifies rejectionrequest invalidation of theFARP Response command Accept (ACC) Reply Sequence Signifies successful completionservice parameters and Port_ID of theFARP Response command Rajagopal, Bhagawat, Rickard [Page 16] Internet-Draft IPrecipient N_Port. The level of initialization andARP over Fibre Channel June 22 1998subsequent validation and recovery reported to the upper (FC-4) layers is implementation-specific. In general, an explicit Logout (LOGO) shall be sent whenever the FC- Layer mapping between the Port_ID and WW_PN of a remote port is removed. Theformateffect of power-up or re-boot on theFARP Response payloadmapping tables isas follows: +-----------------------------------------+---------+ | FARP Response Payload | | +-----------------------------------------+---------+ | Field | Size | | | (Bytes) | +-----------------------------------------+---------+ | 0x54-01-00-00 | 4 | +-----------------------------------------+---------+ | Reserved | 1 | +-----------------------------------------+---------+ | Port_ID of Responder | 3 | +-----------------------------------------+---------+ |WWP_N of Originator (FARP Request) | 8 | +-----------------------------------------+---------+ |WWN_N of Originator (FARP Request) | 8 | +-----------------------------------------+---------+ |WWP_N of Responder | 8 | +-----------------------------------------+---------+ |WWN_N of Responder | 8 | +-----------------------------------------+---------+ Accept Payload: The format ofoutside theFARP Accept payload is as follows: +-----------------------------------------+---------+ | FARP Response Accept Payload | | +-----------------------------------------+---------+ | Field | Size | | |(Bytes) | +-----------------------------------------+---------+ | x02-00-00-00 | 4 | +-----------------------------------------+---------+ 6.0scope of this specification. 6.2 FClayerLayer Address ValidationAt all times, the <MAC Address, Port_ID> mapping has to be validated before use. There are many events that can invalidate this mapping. The following discussion addresses conditions when suchin a Point-to-Point Topology No validation isrequired. 6.1 General Discussion After a link interruption occurs, the Port_ID ofrequired after LR. In aport may change. After the interruption, the Port_IDspoint-to-point topology, NOS/OLS causes implicit logout ofall other ports that have previously performed PLOGI (N_Port Login) with thiseach portmay have changed,andits own Port_ID may have changed. Because of this, address validation is requiredafter aLIP inNOS/OLS, each port must perform aloop topology [7]or after NOS/OLSPLOGI [2]. 6.3 FC Layer Address Validation in apoint-to-point topology [6]. Rajagopal, Bhagawat, Rickard [Page 17] Internet-Draft IP and ARP over Fibre Channel June 22 1998 Port_IDs will not change as a result of Link Reset(LR),thus address validation is not required. In addition to actively validating devices afterPrivate Loop Topology After alink interruption, ifLIP, a portreceivesshall not transmit anyFC-4 data frames (other than broadcast frames), from a port that is not currently logged in, then it shall send an explicit Extended Link Service (ELS) Request logout (LOGO) command to that port. ELS commands (Requests and Replies) are used by an N_Port to solicit a destination port (F_Port or N_Port) to perform some link-level function or service.) The LOGO Request is used to request invalidation of the service parameters and Port_ID of the recipient N_Port. The level of initialization and subsequent validation and recovery reported to the upper (FC-4) layers is implementation-specific. In general, an explicit Logout (LOGO) shall be sent whenever the FC- Layer mapping between the Port_ID and WWP_N of a remote port is removed. The effect of power-up or re-boot on the mapping tables is outside the scope of this specification. 6.2 FC Layer Address Validation in a Point-to-Point Topology No validation is required after LR. In a point-to-point topology, NOS/OLS causes implicit logout of each port and after a NOS/OLS, each port must perform a PLOGI [2]. 6.3 FC Layer Address Validation in a Private Loop Topology After a LIP, a port shall not transmit any linklink data to another port until the address of the other port has been validated. The validation consists of completing either ADISC or PDISC. (See Appendix A) ADISC (Address Discovery) is an ELS command for discovering the hard addresses - the 24-bitNL_portidentifier- ofN_PortsNL_Ports [5], [6]. PDISC (Discover Port) is an ELS command for exchanging service parameters without affecting login state [5], [6]. Rajagopal, Bhagawat, Rickard [Page 17] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 As a requester, this specification prohibits PDISC and requires ADISC. As a responder, an implementation may need to respond to both ADISC and PDISC for compatibility with other FC specifications. If the three addresses, Port_ID,WWP_N, WWN_N,WW_PN, WW_NN, exactly match the values prior to the LIP, then any active exchanges may continue. If any of the three addresses have changed, then the node must beRajagopal, Bhagawat, Rickard [Page 18] Internet-Draft IP and ARP over Fibre Channel June 22 1998 either implicitly orexplicitly logged out [4], [5]. If a port's N_Port ID changes after a LIP, then all active Port-ID to WW_PN mappings at this port must be explicitly logged out. 6.4 FC Layer Address Validation in a Public Loop TopologyAfter a LIP, each public loop port shall not transmit any frame until it receives the FAN ELS from the fabric [4]. AA FAN (Fabric Address Notification) ELS command is sent by the fabric to all known previously logged in ports following an initialization event. Therefore, after a LIP, hosts may wait for this notification to arrive or they may perform a FLOGI. TheWWP_NWW_PN andWWN_NWW_NN of the fabric FL_Port contained in the FAN ELS or FLOGI response must exactly match the values before the LIP. In addition, the AL_PA obtained by the port must be the same as the one before the LIP. If the above conditions are met, the port may resume all exchanges. If not, then FLOGI (Fabric login) must be performed with the fabric and all nodes must beeither implicitly orexplicitly logged out. A public loop device will have to perform the private loop authentication to any nodes on the local loop which have an Area + Domain Address == 0x00-00-XX 6.5 FC Layer Address Validation in a Fabric Topology No validation is required after LR (link reset). After NOS/OLS, a port must perform FLOGI. If, after FLOGI, the S_ID of the port, theWW Port NameWW_PN of the fabric, and theWWN_NWW_NN of the fabric are the same as before the NOS/OLS, then the port may resume all exchanges. If not, all nodes must beeither, implicitly orexplicitly, logged out [2]. 7. Exchange Management 7.1 Exchange Origination FC Exchanges shall be established to transfer data between ports. Frames on IP exchanges shall not transfer Sequence Initiative. 7.2 Exchange Termination With the exception of the recommendations in AppendixC,B, "Reliability Rajagopal, Bhagawat, Rickard [Page 18] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 in Class 3", the mechanism for aging or expiring exchanges based on activity, timeout, or other method is outside the scope of this document. Exchanges may be terminated by either port. The Exchange Originatorshall normallymay terminate Exchanges by setting the LS bit, following normal FC standard FC-PH [2] rules. This specification prohibits the use of the NOP ELS with LS set for Exchange termination.Rajagopal, Bhagawat, Rickard [Page 19] Internet-Draft IP and ARP over Fibre Channel June 22 1998Exchanges may be torn down by the Exchange Originator or Exchange Responder by using the ABTS_LS protocol. The use of ABTS_LS for terminating aged exchanges or error recovery is outside the scope of this document. The termination of IP exchanges by Logout is discouraged, since this may terminate active exchanges on other FC-4s. 8. Summary of Supported Features Note: 'Required' means the feature support is mandatory, 'Prohibited' means the feature support is not allowed, 'Allowed' means the feature support is optional, and 'Settable' means support is as specified in the relevant standard. 8.1 FC-4 Header (Note 1) +--------------------------------------------------------------------+ | Feature | Support | Notes | +--------------------------------------------------------------------+ | Type Code ( = 5) ISO8802-2LLC/SLLC/SNAP | Required | 2 | | Network Headers | Required | 3 | | Other Optional Headers | Prohibited | | +--------------------------------------------------------------------+ Notes: 1. This table applies only to FC-4 related data, such as IP and ARP packets. This table does not apply to link services and other non-FC-4 sequences (PLOGI, for example) that must occur for normal operation. 2. The TYPE field in the FC Header (Word 2 bits 31-24) must indicate ISO 8802-2 LLC/SNAP Encapsulation (Type 5). This revision of the document focuses solely on the issues related to running IP and ARP over FC. All other issues are outside the scope of this document, including full support for IEEE 802.2 LLC. 3. DF_CTL field (Word 3, bits 23-16 of FC-Header)must indicate the presence of a Network Header (0010 0000) on the First logical Frame of FC-4 sequences. 8.2 R_CTL (FC-Header Word 0, bits31-14)31-24) Rajagopal, Bhagawat, Rickard [Page 19] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 +--------------------------------------------------------------------+ | Feature | Support | Notes | +--------------------------------------------------------------------+ | Information Category (R_CTL Routing): | | | | FC-4 Device Data | Required | 1 | | Extended Link Data | Required | 2 | | FC-4 Link Data | Prohibited | | | Video Data | Prohibited | | | Basic Link Data | Required | 3 | | Link Control | Required | 4 | | R_CTL information | | | | Uncategorized | Prohibited | |Rajagopal, Bhagawat, Rickard [Page 20] Internet-Draft IP and ARP over Fibre Channel June 22 1998| Solicited Data | Prohibited | | | Unsolicited Control | Required | 2 | | Solicited Control | Required | 2 | | Unsolicited Data | Required | 1 | | Data Descriptor | Prohibited | | | Unsolicited Command | Prohibited | | | Command Status | Prohibited | | +--------------------------------------------------------------------+ Notes: 1. This is required for FC-4 (IP and ARP) packets - Routing bits of R_CTL field must indicate Device Data frames (0000). - Information Category of R_CTL field must indicate Unsolicited Data (0100). 2. This is required for Extended Link Services. 3. This is required for Basic Link Services. 4. This is required for Link Control frames. 8.3 F_CTL (FC-Header Word 2, bits 23-0) +--------------------------------------------------------------------+ | Feature | Support | Notes | +--------------------------------------------------------------------+ | Exchange Context | Settable | | | Sequence Context | Settable | | | First / Last / End Sequence (FS/LS/ES) | Settable | | | Chained Sequence | Prohibited | | | Sequence Initiative (SI) | Settable | 1 | | X_ID Reassigned / Invalidate | Prohibited | | | Unidirectional Transmit | Settable | | | Continue Sequence Condition | Required | 2 | | Abort Seq. Condition -continue and single seq.| Required | 3 | | Relative Offset - Unsolicited Data | Settable | 4 | | Fill Bytes | Settable | | +--------------------------------------------------------------------+ Notes: 1. For FC-4 frames, each N_Port shall have a dedicatedX_IDOX_ID for sending data to each N_Port in the network and a dedicatedX_IDRX_ID for receiving data from each N_Port as well. Exchanges are used in a unidirectional mode, thus setting sequence initiative is not valid for FC-4 frames. Sequence initiative Rajagopal, Bhagawat, Rickard [Page 20] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 is valid when using Extended Link Services. 2. This field is required to be 00, no information. 3. Sequence error policy is requested by an exchange originator in the F_CTL Abort Sequence Condition bits in the first data frame of the exchange. For classes 1 and 2, ACK frame is required to be "continuous sequence". 4. Relative offset prohibited on all other types (InformationRajagopal, Bhagawat, Rickard [Page 21] Internet-Draft IP and ARP over Fibre Channel June 22 1998Category) of frames. 8.4 Sequences +---------------------------------------------------------------------+ | Feature | Support |Notes | +---------------------------------------------------------------------+ | Class 2 open sequences / exchange | 1 | 1 | | Length of seq. not limited by end-to-end credit | Required | 2 | | Maximum sequence size - IP sequences | 65536 | 3 | | Maximum sequence size - ARP sequences | 532 | 4 | | Capability to receive sequence of maximum size | Allowed | 5 | | Sequence Streaming | Prohibited | 6 | | Stop Sequence Protocol | Prohibited | | | ACK_0 support | Allowed | 7 | | ACK_1 support | Required | 7 | | ACK_N support | Prohibited | | | Class of Service for transmitted sequences | 1, 2 or 3 | 8 | | Continuously Increasing Sequence Count | Allowed | 9,10 | +---------------------------------------------------------------------+ Notes: 1. Only one active sequence per exchange is allowed. 2. A sequence initiator shall be capable of transmitting sequences containing more frames than the available credit indicated by a sequence recipient at login. FC-PH [2] end-to end flow control rules will be followed when transmitting such sequences. 3. Maximum sequence size is6553665535 bytes. Thus if the maximum IP packet size (MTU) is limited to 65280 bytes, then after accounting for 8 bytes(65536of LLC/SNAP, we stll have (65535 -256 bytes65280 -8 = 247 bytes) for headeroverhead).overhead. 4. Maximum size ARP packet is 532 bytes (including LLC/SNAP headers). 5. Some OS environments may not handle the max MTU of65536.65535. It is up to the administrator to configure the Max MTU for all systems. 6. All class 3 sequences are assumed to be non-streamed. 7. Only applies for Class 1 and 2. Use of ACK_1 is default, ACK_0 used if indicated by sequence recipient at login. Rajagopal, Bhagawat, Rickard [Page 21] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 8. The administrator configured class of service is used, except where otherwise specified (e.g. Broadcasts are always sent in class 3). 9. Review AppendixC,B, "Reliability in Class 3". 10. The first frame of the first sequence of anew exchange must have SEQ_CNT = 0 [2].Rajagopal, Bhagawat, Rickard [Page 22] Internet-Draft IP and ARP over Fibre Channel June 22 19988.5 Exchanges +--------------------------------------------------------------------+ | Feature | Support | Notes | +--------------------------------------------------------------------+ | X_ID interlock support | Allowed | 1 | | OX_ID=FFFF | Prohibited | | | RX_ID=FFFF | Allowed | 2 | | Action if no exchange resources available | P_RJT | 3 | | Long Lived Exchanges | Allowed | 4 | | Reallocation of Idle Exchanges | Allowed | | +--------------------------------------------------------------------+ Notes: 1. Only applies to Classes 1 and 2, supported by the exchange originator. A Port shall be capable of interoperating with another Port that requires X_ID interlock. The exchange originator facility within the Port shall use the X_ID Interlock protocol in such cases. 2. An exchange responder is not required to assign RX_IDs. If a RX_ID of FFFF is assigned, it is identifying exchanges based on S_ID / D_ID / OX_ID only. 3. In Classes 1 and 2, a Port shall reject a frame that would create a new exchange with a P_RJT containing reason code "Unable to establish exchange". In Class 3, the frame would be dropped. 4. When an exchange is created between 2 Ports for IP/ARP data, it remains active while the ports are logged in with each other. An exchange shall not transfer Sequence Initiative (SI). Broadcasts and ELS commands may use short lived exchanges. 8.6 ARP +--------------------------------------------------------------------+ | Feature | Support | Notes | +--------------------------------------------------------------------+ | ARP Server Support | Prohibited | 1 | | Response to ARP requests | Required | 2 | | ARP requests transmitted as broadcast message | Required | | | Class of Service for ARP requests | 3 | 3 | | Class of Service for ARP replies | 1, 2 or 3 | 4 | +--------------------------------------------------------------------+ Notes: 1. Well-known Address FFFFFC is not used for ARP requests. Frames Rajagopal, Bhagawat, Rickard [Page 22] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 from Well-known Address FFFFFC are not considered to be ARP frames. Broadcast support is required for ARP. 2. The IP Address is mapped to a specific MAC address with ARP.Rajagopal, Bhagawat, Rickard [Page 23] Internet-Draft IP and ARP over Fibre Channel June 22 19983. An ARP request is a broadcast message, thus Class 3 is always used. 4. An ARP reply is a normal sequence, thus the administrator configured class of service is used. 8.7 Extended Link Services (ELS) +--------------------------------------------------------------------+ | Feature | Support | Notes | +--------------------------------------------------------------------+ | Class of service for ELS commands / responses | 1,2 or 3 | 1 | | Explicit N-Port Login | Required | | | Explicit F-Port Login | Required | | | FLOGI ELS command | Required | | | PLOGI ELS command | Required | | | ADISC ELS command | Required | | | PDISC ELS command | Allowed | 2 | | FAN ELS command | Required |35 | | LOGO ELS command | Required | | | FARP-REQ ELS command | Allowed | 3 | | FARP-RES ELS command | Required | 3 | | Other ELS command support | Allowed | 4 |+--------------------------------------------------------------------++-----------------------------------------------+------------+-------+ Notes: 1. The administrator configured class of service is used. 2. PDISC is prohibited as requester. ADISC should be used instead. As a responder, an implementation may need to respond to both ADISC and PDISC for compatibility with other specifications. 3.FANFARP is allowed as Requester but requiredin a public loop environment.as Responder 4. If other ELS commands are received an LS_RJT may be sent. NOP is not required by this specification, and should not be used as a mechanism to terminate exchanges. 5. Required for FL_Ports 8.8 Login Parameters Unless explicitly noted here, a compliant implementation shall use the login parameters as described in [4]. 8.8.1 Common Service Parameters - FLOGI - FC-PH Version, lowest version may be 0x09 to indicate 'minimum 4.3'. Rajagopal, Bhagawat, Rickard [Page 23] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 - Can't use BB_Credit=0 for N_Port on a switched Fabric (F_Port). 8.8.2 Common Service Parameters - PLOGI - FC-PH Version, lowest version may be 0x09 to indicate 'minimum 4.3'.Rajagopal, Bhagawat, Rickard [Page 24] Internet-Draft IP and ARP over Fibre Channel June 22 1998- Can't use BB_Credit=0 for N_Port in a Point-to-Point configuration - Random Relative Offset is allowed. - Note that the 'Receive Data Field Size' fields specified in the PLOGI represent both optional headers and payload. - The MAC Address can therefore be extracted from the 6 lower bytes of theWWP_NWW_PN field (when the IEEE 48-bit Identifier format is chosen as the NAA) during PLOGI or ACC payload exchanged during Fibre Channel Login [2]. - The MAC Address can also be extracted from theWWP_NWW_PN field in the Network Header during ADISC (and ADISC ACC), or PDISC (and PDISC ACC). 8.8.3 Class3Service Parameters - PLOGI - Discard error policy only. ACKNOWLEDGEMENT This specification is based on FCA IP Profile, Version 3.3. The FCA IP Profile was a joint work of the Fibre Channel Association (FCA) vendor community. The following companies and organizations have contributed to the creation of the FCA IP Profile: Adaptec, Ancor, Brocade,Clarion,Clariion, Crossroads, emf Associates, Emulex, Finisar, Gadzoox, Hewlett Packard, Interphase, Jaycor, LLNL, McData, Migration Associates, Orca Ssytems, Prisa, Q-Logic, Symbios, Systran, Tektronix, Univ. of Minnesota, Univ. of New Hamshire. REFERENCES [1] FCA IP Profile, Revision2.3,3.3, May 15, 1997 [2] Fibre Channel Physical and Signaling Interface (FC-PH) , ANSI X3.230-1994 [3] Fibre Channel Link Encapsulation (FC-LE), Revision 1.1, June 26, 1996 [4] Fibre Channel Fabric Loop Attachment (FC-FLA), Rev. 2.4, October 21, 1996 [5] Fibre Channel Private Loop SCSI Direct Attach (FC-PLDA), Rev.1.7, October 7, 1996[6] Fibre ChannelRajagopal, Bhagawat, Rickard [Page 24] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 [6] Fibre Channel Physical and Signaling Interface-2 (FC-PH-2), Rev. 7.4, ANSI X3.297-1996 [7] Fibre Channel Arbitrated Loop (FC-AL), ANSI X3.272-1996 [8] Postel, J. and Reynolds, J., "A standard for the Transmission ofRajagopal, Bhagawat, Rickard [Page 25] Internet-Draft IP and ARP over Fibre Channel June 22 1998IP Datagrams over IEEE 802 Networks". RFC 1042, ISI, Feb, 1988 [9] Plummer, D. "An Ethernet Address Resolution Protocol -or- Converting Network Addresses to 48-bit Ethernet Address for Transmission on Ethernet Hardware", STD 37, RFC 826, MIT, Nov 1982. [10] FCSI IP Profile, FCSI-202, Revision 2.1, September 8, 1995 [11] Fibre Channel Physical and Signaling Interface -3 (FC-PH-3), Rev. 9.1, ANSI X3.xxx-199x [12] Fibre Channel-The Basics, "Gary R. Stephens and Jan V. Dedek", Ancot Corporation [13] Fibre Channel -Gigabit Communications and I/O for Computers Networks "Alan Benner", McGraw-Hill, 1996, ISBN 0-07-005669-2 [14] [ns008.pdf on ftp.network.com] AUTHORS' ADDRESSES Murali Rajagopal Gadzoox Networks, Inc. 711 Kimberly Avenue, Suite 100 Placentia, CA 92870 Phone: +1 714 577 6805 Fax: +1 714 524 8508 Email: murali@gadzoox.com Raj Bhagwat Gadzoox Networks, Inc. 711 Kimberly Avenue, Suite 100 Placentia, CA 92870 Phone: +1 714 577 6806 Fax: +1 714 524 8508 Email: raj@gadzoox.com Wayne Rickard Gadzoox Networks, Inc. 711 Kimberly Avenue, Suite 100 Placentia, CA 92870 Phone: +1 714 577 6803 Fax: +1 714 524 8508 Email: wayne@gadzoox.com APPENDIX - A Rajagopal, Bhagawat, Rickard [Page 25] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 FIBRE CHANNEL OVERVIEW A.1 Brief Tutorial FC standard [2] defines 4 "levels" (not layers) for its protocol description: FC-0, FC-1, FC-2, FC-3, and FC-4. The first three levels (FC-0, FC-1, FC-2)Rajagopal, Bhagawat, Rickard [Page 26] Internet-Draft IP and ARP over Fibre Channel June 22 1998are largely concerned with the physical formatting and control aspects of the protocol. FC-3 is architecturally defined but not unspecified at this time. FC-4 is meant for supporting profiles of higher protocols such as IP and Small ComputerSerialSystem Interface (SCSI) and supports a relatively small set of higher level protocols compared to LAN protocols such as IEEE 802.3. FC devices are called "Nodes", each of which has at least one "Port" to connect to other ports. A Node may be a workstation, a disk drive or disk array, a camera, a display unit, etc. The set of hardware components, andtransreceivers,transceivers, connecting two or more node ports is called a topology. A "Link" is two unidirectional paths flowing in opposite directions and connecting two Ports within adjacent Nodes. FC Nodes communicate usingthesehigher layer protocols such as SCSI and IPover FCand are configured to operate using one of the following networking topologies: - Point-to-Point - Private Loop - Public Loop (attachment to a Fabric) - Fabric The point-to-point is the simplest of the four topologies, where only two nodes communicate with each other. The private loop may connect a number of devices (max 126) in a logical ring much like Token Ring and is distinguished from a public loop by the absence of a Fabric Node participating in the loop. The Fabric topology is a switched network where any attached node can communicate with any other. Table below summarizes the usage of port types depending on its location[12]:[12]. Note that E-Port is not relevant to any discussion in this specification but is included below for completeness. +-----------+-------------+-----------------------------------------+ | Port Type | Location | Topology Associated with | +-----------+-------------+-----------------------------------------+ | N_Port | Node | Point-to-Point or Fabric | +-----------+-------------+-----------------------------------------+ | NL_Port | Node |In N_Port mode -Point-to-Point or Fabric | | | |In NL_Port mode - Arbitrated Loop | +-----------+-------------+-----------------------------------------+ | F_Port | Fabric | Fabric | +-----------+-------------+-----------------------------------------+ | FL_Port | Fabric | In F_Port mode - Fabric | | | | In FL_Port mode - Arbitrated Loop | +-----------+-------------+-----------------------------------------+ | E_Port | Fabric | Internal Fabric Expansion | +-----------+-------------+-----------------------------------------+| G_Port | Fabric | In F_Port mode - Fabric | | | | In E_Port mode - Internal Fabric Expan.| +-----------+-------------+-----------------------------------------+ | GL_Port | Fabric | In F_Port mode - Fabric | | | | In FL_Port mode - Abritrated Loop | | | | In E_Port mode - Internal Fabric Expan. | +-----------+-------------+-----------------------------------------+Rajagopal, Bhagawat, Rickard [Page27]26] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998 A.2 Fibre Channel Header Fields Fibre Channel Frame Header, Network Header, and payload carrying IP Packet +---+----------------+----------------+----------------+--------------+ |Wrd| <31:24> | <23:16> | <15:08> | <07:00> | +---+----------------+----------------+----------------+--------------+ |0 |RTCLR_CTL | D_ID | +---+----------------+----------------+----------------+--------------+ |1 |RSVDCS_CTL | S_ID | +---+----------------+----------------+----------------+--------------+ |2 | TYPE | F_CTL | +---+----------------+----------------+----------------+--------------+ |3 | SEQ_ID | DF_CTL | SEQ_CNT | +---+----------------+----------------+----------------+--------------+ |4 | OX_ID | RX_ID |+---+----------------+----------------+----------------+--------------++---+--------+-------+----------------+----------------+--------------+ |5 | Parameter (Control or Relative Offset for Data ) | +---+-----------------------------------------------------------------+ |6 | NAA | Network_Dest_Address(MSB)(Hi order bits) |+---+----------------+----------------+----------------+--------------+ |6+---+--------+-------+----------------+----------------+--------------+ |7 | Network_Dest_Address(LSB)(Lo order bits) |+---+----------------+----------------+----------------+--------------+ |7+---+--------+-------+----------------+----------------+--------------+ |8 | NAA | Network_Src_Address(MSB)(Hi order bits) |+---+----------------+----------------+----------------+--------------+ |8+---+--------+-------+----------------+----------------+--------------+ |9 | Network_Src_Address(LSB)(Lo order bits) | +---+----------------+----------------+----------------+--------------+|9|10 | DSAP | SSAP | CTRL | OUI | +---+----------------+----------------+----------------+--------------+|10|11 | OUI | PID | +---+----------------+----------------+----------------+--------------+|11|12 | IP Packet Data | +---+----------------+----------------+----------------+--------------+|12|13 | ... | +---+----------------+----------------+----------------+--------------+ The FC header as shown in the above diagrams contains routing and other control information to manage frames, sequences, and exchanges. The frame header is sent as 6 transmission words immediately following an SOF delimiter and before the data field. D_ID and S_ID: FC uses destination address routing [12], [13]. Frame routing in a point-to-point topology is trivial. For the Arbitrated Loop topology, with the destination NL_Port on the same AL, the source port must pick the destination port, determine its AL Physical Address, and "Open" the destination port. The frames must pass through other NL_Ports or the FL_Port on the loop between the source and destination, but these ports do not capture the frames. They simply repeat and transmit the frame. Either communicating port may "Close" the circuit. Rajagopal, Bhagawat, Rickard [Page 27] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 When the destination port is not on the same AL, the source NL_Port must open the FL_Port attached to a Fabric. Once in theRajagopal, Bhagawat, Rickard [Page 28] Internet-Draft IP and ARP over Fibre Channel June 22 1998Fabric, the Fabric routes the frames again to the destination. In a Fabric topology, the Fabric looks into the frame header, extracts the destination address (D_ID), searches its own routing tables, and sends the frame to the destination port along the path chosen. The process of choosing a path may be performed at each fabric until the F_Port attached to the destination N_Port is reached. R_CTL (Routing Control) and TYPE(data structure): Frames for each FC-4 can be easily distinguished from the others at the receiving port using the R_CTL (Routing Control) and TYPE (data structure) fields in the frame header. The R_CTL has two sub-fields: Routing bits and Information category. The Routing bits sub-field has specific values that mean FC-4 data follows and the Information Category tells the receiver the "Type" of data contained in the frame. The R_CTL and TYPE code points are shown in the diagrams. Other Header fields: F_CTL (Frame Control) and SEQ_ID (Sequence Identification), SEQ_CNT (Sequence Count), OX_ID (Originator exchange Identifier), RX_ID (Responder exchange Identifier), and Parameter fields are used to manage the contents of a frame, and mark information exchange boundaries for the destination port. F_CTL(Frame Control): The FC_CTL field is a 3-byte field that contains information relating to the frame content. Most of the other frame header fields are used for frame identification. Among other things, bits in this field indicate the first sequence, last sequence, or end sequence. Sequence Initiative bit is used to pass control of the next sequence in the exchange to the recipient. SEQ_ID (Sequence Identifier) and SEQ_CNT (Sequence Count): This is used to uniquely identify sequences within an Exchange. The <S_ID, D_ID, SEQ_ID> uniquely identifies any active sequence. SEQ_CNT is used to uniquely identify frames within a Sequence to assure sequentiality of frame reception, and to allow unique correlation of link control frames with their related data frames. Originator Exchange Identifier (OX_ID) and Responder Exchange Identifier (RX_ID): The OX_ID value provides association of frames with specific Exchanges originating at a particular N_Port. The RX_ID field Rajagopal, Bhagawat, Rickard [Page 28] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 provides the same function that the OX_ID provides for the Exchange Originator. The OX_ID is meaningful on the Exchange Originator, and the RX_ID is meaningful on the Responder.Rajagopal, Bhagawat, Rickard [Page 29] Internet-Draft IP and ARP over Fibre Channel June 22 1998DF_CTL (Data Field Control): The DF_CTL field specifies the presence or absence of optional headers between the Frame header and Frame Payload PARAMETER: The Parameter field has two meanings, depending on Frame type. For Link Control Frames, the Parameter field indicates the specific type of link Link Control frame. For Data frames, this field contains the Relative Offset value. This specifies an offset from an Upper Layer Protocol buffer from a base address. Code Points for FC Frame withIP/ARPIP packet Data +---+----------------+----------------+----------------+--------------+ |Wrd| <31:24> | <23:16> | <15:08> | <07:00> | +---+----------------+----------------+----------------+--------------+ |10 | 0x04 | D_ID | +---+----------------+----------------+----------------+--------------+ |21 | 0x00 | S_ID | +---+----------------+----------------+----------------+--------------+ |32 | 0x05 | F_CTL | +---+----------------+----------------+----------------+--------------+ |43 | SEQ_ID | 0x20 | SEQ_CNT | +---+----------------+----------------+----------------+--------------+ |54 | OX_ID | RX_ID | +---+----------------+----------------+----------------+--------------+ | 5 | 0xXX-XX-XX-XX Parameter Relative Offset | +---+------+----------------------------------------------------------+ | 6 | 0001 |0x00-00-000x000 | Dest. MAC (Hi order bits) |+---+----------------+----------------+----------------+--------------++---+------+---------+----------------+----------------+--------------+ | 7 | Dest. MAC(LSB)(Lo order bits) |+---+----------------+----------------+-------------------------------++---+------+----------+----------------+------------------------------+ | 8 | 0001 |0x00-00-000x000 | Src. MAC (Hi order bits) |+---+----------------+----------------+----------------+--------------++---+------+---------+----------------+----------------+--------------+ | 9 | Src. MAC(LSB)(Lo order bits) | +---+----------------+----------------+----------------+--------------+ |10 | 0xAA |0x000xAA | 0x03 | 0x00 | +---+----------------+----------------+----------------+--------------+ |11 | 0x00-00 | 0x08-00 | +---+----------------+----------------+----------------+--------------+ |12 |IP/ARPIP Packet Data | +---+----------------+----------------+----------------+--------------+|13 ||13| ... | +---+----------------+----------------+----------------+--------------+ The Code Points for FC Frames with ARP packets are very similar to IP packets with the exception of PID value in Word 11 which is set to 0x08-06. Also, the Network Header as shown above appears only in the Rajagopal, Bhagawat, Rickard [Page 29] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 first logical FC Sequence carrying IP. In the case, where FC frames carry ARP packets it is always present because these are single frame sequences. A.3 Acronyms and Glossary of FC Terms It is assumed that the reader is familiar with the terms and acronyms used in theFC protocol specification [2]. The following is provided for easy reference. A.3.1 Acronyms First Frame: The frameFC protocol specification [2]. The following is provided for easy reference. A.3.1 Acronyms First Frame: The frame that contains the SOFi field. This means a logical first and may not necessarily be the first frame temporally received in a sequence. Code Point: The coded bit pattern associated with control fields in frames or packets. PDU: Protocol Data Unit ABTS_LS: Abort Sequence Protocol - Last Sequence. A protocol for aborting an exchange based on the ABTS recipient setting the Last_Sequence bit in the BA_ACC ELS to the ABTS ADISC: Discover Address. An ELS for discovering the Hard Addresses (the 24 bit NL_Port Identifier) of N_Ports D_ID: Destination ID ES: End sequence. This FCTL bit in the FC header indicates this frame is the last frame of the sequence. FAN: Fabric Address Notification. An ELS sent by the fabric to all known previously logged in ports following an initialization event. FLOGI: Fabric Login. LIP: Loop Initialization. A primitive sequence used by a port to detect if it is part of a loop or to recover from certain loop errors. Link: Two unidirectional paths flowing in opposite directions and connecting two Ports within adjacent Nodes. LOGO: Logout. LR: Link reset. A primitive sequence transmitted by a port to initiate the link reset protocol or to recover from a link timeout. LS: Last sequence of Exchange. This FCTL bit in the FC header indicates the sequence is the last sequence of the exchange. Network Address Authority: A 4-bit field specified in Network Headers that distinguishes between various name registration authorities that Rajagopal, Bhagawat, Rickard [Page 30] Internet-Draft IP and ARP over Fibre Channel Sep 18 1998 may be used to identify the WW_PN and the WW_NN. NAA=b'0001' indicates IEEE-48-bit MAC addresses Node: A collection of one or more Ports identified by a unique World Wide Node Name (WW Node Name). NOS: Not Operational. A primitive sequence transmitted to indicate that the port transmitting this sequence has detected a link failure or is offline, waiting for OLS to be received. OLS: Off line. A primitive sequence transmitted to indicate that the port transmitting this sequence is either initiating the link initialization protocol, receiving and recognizing NOS, or entering the offline state. PDISC: Discover Port. An ELS for exchanging Service Parameters without affecting login state. Primitive Sequence: A primitive sequence is an Ordered Set that is transmitted repeatedly and continuously. Private Loop Device: A device that does not attempt fabric login (FLOGI) and usually adheres to PLDA. The Area and Domain components of the NL_Port ID must be 0x0000. These devices cannot communicate with any port not in the local loop. Public Loop Device: A device whose Area and Domain components of the NL_Port ID cannot be 0x0000. Additionally, to be FLA compliant, the device must attempt to open AL_PA 0x00 and attempt FLOGI. These devices communicate with devices on the local loop as well as devices on the other side of a Fabric. Port: The transmitter, receiver and associated logic at either end of a link within a Node. There may be multiple Ports per Node. Each Port is identified by a unique Port_ID, which is volatile, and a unique World Wide Port Name (WW Port Name), which is unchangeable. In this document, the term "port" may be used interchangeably with NL_Port or N_Port. Port_ID: Fibre Channel ports are addressed by unique 24-bit Port_IDs. In a Fibre Channel frame header, the Port_ID is referred to as S_ID (Source ID) to identify the port originating a frame, and D_ID to identify the destination port. The Port_ID of a given port is volatile (changeable). The mechanisms through which a Port_ID may change in a Fibre Channel topology are outside the scope of this document. PLOGI: Port Login. SI: Sequence Initiative World Wide Port_Name (WW_PN): Fibre Channel requires each Port to have an unchangeable WW_PN. Fibre Channel specifies a Network Address Authority (NAA) to distinguish between the various name registration authorities thatcontainsmay be used to identify theSOFi field. This means a logical firstWW_PN. A 4-bit NAA identifier, 12-bit field set to 0x0 andmayan IEEE 48-bit MAC address Rajagopal, Bhagawat, Rickard [Page30]31] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998not necessarily be the first frame temporally received intogether make this asequence. Code Point: The coded bit pattern associated64-bit field. World Wide Node_Name (WW_NN): Fibre Channel identifies each Node withcontrol fields in frames or packets. PDU: Protocol Data Unit ABTS_LS: Abort Sequence Protocol - Last Sequence. A protocol for aborting an exchange based on the ABTS recipient setting the Last_Sequence bit in the BA_ACC ELS toa unchangeable WW_NN. In a single port Node, theABTS ADISC: Discover Address. An ELS for discoveringWW_bNN and theHard Addresses (the 24 bit NL_Port Identifier) of N_Ports D_ID: DestinationWW_PN may be identical. APPENDIX - B B.1 RELIABILITY IN CLASS 3 Problem: Sequence IDES: End sequence. This FCTL bitreuse inthe FC header indicates this frame is the lastClass 3 can conceivably result in missing frameof the sequence. FAN: Fabric Address Notification. An ELS sent byaliasing with no corresponding detection at the FC2 level. Prevention: This specification requires one of thefabric to all known previously logged in portsfollowingan initialization event. LIP: Loop Initialization. A primitive sequence used by a port to detectmethods if Class 3 is used. - Continuously increasing Sequence Count (new Login Bit) - both sides must set When an N_Port sets the PLOGI login bit for continuously increasing SEQ_CNT, it ispart ofguaranteeing that it will transmit all frames within an exchange using aloop or to recover from certain loop errors. LR: Link reset. A primitive sequence transmitted bycontinuously increasing SEQ_CNT (see description below). - After using all SEQ_IDs (0-255) once, must start aport to initiate the link reset protocol or to recover fromnew Exchange. It is recommended that alink timeout. LS: Last sequenceminimum ofExchange. This FCTL bit in the FC header indicates the sequence4 Exchanges be used before an OX_ID can be reused. - Note: If an implementation is not checking thelast sequence ofOX_ID when reassembling sequences, theexchange. NOS: Not Operational. A primitive sequence transmittedproblem can still occur. Cycling through some number of SEQ_IDs, then jumping toindicate that the port transmitting this sequence has detectedalink failure or is offline, waiting for OLS to be received. OLS: Off line. A primitive sequence transmitted to indicate thatnew exchange does not solve theport transmitting this sequence is either initiatingproblem. SEQ_IDs must still be unique between two N_Ports, even across exchanges. - Use only single-frame Sequences. B.2 CONTINUOUSLY INCREASING SEQ_CNT This method allows thelink initialization protocol, receiving and recognizing NOS, or enteringrecipient to check incoming frames, knowing exactly what SEQ_CNT value to expect next. Since theoffline state. PDISC: Discover Port. An ELSSEQ_CNT will not repeat forexchanging Service Parameters without affecting login state. SI: Sequence Initiative FLOGI: Fabric Login. Primitive Sequence: A primitive sequence is an Ordered Set that65,536 frames, the aliasing problem istransmitted repeatedly and continuously. Private Loop Device:significantly reduced. Adevice that does not attempt fabriclogin(FLOGI) and usually adheresbit (PLOGI) is used toPLDA. The Area and Domain componentsindicate that a device always uses a continuously increasing SEQ_CNT, even across transfers ofthe NL_Port ID must be 0x0000. These devices cannot communicatesequence initiative. This bit is necessary for interoperability withany port not insome devices, and it provides other benefits as well. In thelocal loop.FC-PH-3 [11], the following is supported: Word 1, bit 17 - SEQ_CNT (S) 0 = Normal FC-PH rules apply 1 = Continuously Increasing SEQ_CNT Rajagopal, Bhagawat, Rickard [Page31]32] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998Public Loop Device: A device whose Area and Domain components of the NL_Port ID cannot be 0x0000. Additionally, to be FLA compliant, the device must attempt to open AL_PA 0x00 and attempt FLOGI. These devices communicate with devices on the local loop as well as devices on the other side of a Fabric. Link: Two unidirectional paths flowing in opposite directions and connecting two Ports within adjacent Nodes. LOGO: Logout. Node: A collection of one or more Ports identified by a unique World Wide Node Name (WW Node Name). Port: The transmitter, receiver and associated logic at either end of a linkAny N_Port that sets Word 1, Bit 17 = 1, is guaranteeing that it will transmit all frames within an exchange using aNode. There may be multiple Ports per Node.continuously increasing SEQ_CNT. EachPort is identified by a unique Port_ID, which is volatile,exchange shall start with SEQ_CNT = 0 in the first frame, anda unique World Wide Port Name (WW Port Name), which is unchangeable. In this document,every frame transmitted after that shall increment theterm "port" may be used interchangeably with NL_Port or N_Port. Port_ID: Fibre Channel ports are addressedprevious SEQ_CNT byunique 24-bit Port_IDs. In a Fibre Channel frame header,one, even across transfers of sequence initiative. Any frames received from thePort_ID is referredother N_Port in the exchange shall have no effect on the transmitted SEQ_CNT. APPENDIX - C Other Mechansims for FC Layer Mappings Each method should have some mechanism toas S_ID (Source ID)ensure PLOGI has completed successfully before data is sent. A related concern in large networks is limiting concurrent logins toidentifyonly those ports with active IP traffic. C.1 Login on Cached Mapping Information This method insulates theport originating a frame, and D_ID to identifylevel performing LOGIN from thedestination port. The Port_ID of a given portlevel interpreting ARP. It isvolatile (changeable). Themore accommodating of non-ARP mechanismsthrough whichfor building the FC-layer mapping table. 1. Broadcast messages that carry aPort_ID may changeNetwork Header contain the S_ID on the FC-header and WW_PN ina Fibre Channel topology are outsidethescope of this document. PLOGI: Port Login. World Wide Port_Name (WWP_N): Fibre Channel requires each Port to have an unchangeable WWP_N. Fibre Channel specifiesNetwork-header. Caching this information provides aNetwork Address Authority (NAA)correlation of Port_ID todistinguish betweenWW_PN. If thevarious name registration authorities that mayreceived Broadcast message is compliant with this specification, the WW_PN will beused to identifytheWWP_N. A 4-bit NAA identifier, 12-bit field setMAC Address. 2. The WW_PN is "available" if Login has been performed to0x0the Port_ID and flagged. If login has not been performed, the WW_PN is "unavailable". 3. If anIEEE 48-bit MAC address together make this a 64-bit field. World Wide Node_Name (WWN_N): Fibre Channel identifies each Node with a unchangeable WWN_N. Inoutbound packet is destined for asingleportNode,that is "unavailable", theWWN_N andcached information (from broadcast) is used to look up theWWP_N may be identical. APPENDIX - B B.1 RELIABILITY IN CLASS 3 Problem: Sequence ID reuse in Class 3Port_ID. 4. After sending an ELS PLOGI command (Port Login) to the Port (from a higher level entity at the host), waiting for an outbound packet before sending this Port Login conserves resources for only for those ports which wish to establish communication. 5. After Port Login completes (ACC received), the outbound packet canconceivably resultbe forwarded. At this point inmissing frame aliasing with no corresponding detection attime, both ends have theFC2 level. Prevention:necessary information to complete their <IP address, MAC Address, Port_ID> association. C.2 Login on ARP Parsing Thisspecificationmethod performs LOGIN sooner by parsing ARP before passing it up to higher levels for IP/MAC Address correlation. It requiresonea low-level awareness of thefollowing methods if Class 3IP address, and isused. - Continuously increasing Sequence Count (new Login Bit) - boththerefore protocol-specific. 1. When an ARP Broadcast Message is received, the S_ID is extracted from the FC header and the corresponding Network_Source_Address Rajagopal, Bhagawat, Rickard [Page32]33] Internet-Draft IP and ARP over Fibre ChannelJune 22Sep 18 1998sides must set When an N_Port setsfrom thePLOGI login bit for continuously increasing SEQ_CNT, it is guaranteeing that it will transmit all frames within an exchange using a continuously increasing SEQ_CNT (see description below). - After using all SEQ_IDs (0-255) once, must start a new Exchange. ItNetwork Header. 2. The ARP payload isrecommended that a minimum of 4 Exchanges be used before an OX_ID can be reused. - Note: If an implementationparsed to determine if (a) this host isnot checking the OX_ID when reassembling sequences,theproblem can still occur. Cycling through some numbertarget ofSEQ_IDs, then jumping to a new exchange does not solvetheproblem. SEQ_IDs must still be unique between two N_Ports, even across exchanges. - Use only single-frame Sequences. B.2 CONTINUOUSLY INCREASING SEQ_CNT This method allowsARP request (Target IP Address match), and (b) if this host is currently logged in with therecipient to check incoming frames, knowing exactly what SEQ_CNT value to expect next. Sinceport (Port_ID = S_ID) originating theSEQ_CNT will not repeatARP broadcast. 3. The ARP is passed to higher level for65,536 frames, the aliasing problemARP Response generation. 4. If a Port Login issignificantly reduced. A login bit (PLOGI)required, an ELS PLOGI command (Port Login) isusedsent immediately toindicate that a device always uses a continuously increasing SEQ_CNT, even across transfers of sequence initiative. This bitthe Port originating the ARP Broadcast. 5. After Port Login completes, an ARP response can be forwarded. Note that there are two possible scenarios: - The ACC to PLOGI returns before the ARP reply isnecessary for interoperability with some devices,processed andit provides other benefits as well. IntheFC-PH-3 [11], the followingARP Reply issupported: Word 1, bit 17immediately forwarded. -SEQ_CNT (S) 0 = Normal FC-PH rules apply 1 = Continuously Increasing SEQ_CNT Any N_Port that sets Word 1, Bit 17 = 1,The ARP reply isguaranteeing that it will transmit all frames within an exchange using a continuously increasing SEQ_CNT. Each exchange shall start with SEQ_CNT = 0delayed, waiting for ACC (successful Login). 6. At this point in time, both ends have thefirst frame,necessary information to complete their <IP address, MAC Address, Port_ID> association. C.3 Login to Everyone In Fibre Channel topologies with a limited number of ports, it may be efficient to unconditionally login to each port. This method is discouraged in fabric andevery frame transmitted after that shall incrementpublic loop environments. After Port Login completes, theprevious SEQ_CNT by one, even across transfersMAC Address to Port_ID Address tables can be constructed. C.4 Static Table In some loop environments with a limited number ofsequence initiative. Any frames receivedports, a static mapping from a MAC Address to Port_ID (D_ID or AL_PA) may be maintained. The FC layer will always know theother N_Port in the exchange shall have no effectdestination Port_ID based on thetransmitted SEQ_CNT. [draft-ietf-ipfc-00.txt]table. The table is typically downloaded into the driver at configuration time. This method scales poorly, and is therefore not recommended. [draft-ietf-ipfc-fibre-channel-01.txt] [This INTERNET DRAFT expires onDec 22, 1998]Mar 18, 1999] Rajagopal, Bhagawat, Rickard [Page33]34] ----