Bcefe Nutshell

BCEFE in a Nutshell Study Guide for Exam 150-620
Global Education Services Revision 0312
Corporate Headquarters - San Jose, CA USA T: (408) 333-8000 info@brocade.com European Headquarters - Geneva, Switzerland T: +41 22 799 56 40 emea-info@brocade.com Asia Pacific Headquarters - Singapore T: +65-6538-4700 apac-info@brocade.com
2012 Brocade Communications Systems, Inc. All Rights Reserved. Brocade, the Brocade B-weave logo, Fabric OS, File Lifecycle Manager, MyView, Secure Fabric OS, SilkWorm, and StorageX are registered trademarks and the Brocade B-wing symbol and Tapestry are trademarks of Brocade Communications Systems, Inc., in the United States and/ or in other countries. FICON is a registered trademark of IBM Corporation in the U.S. and other countries. All other brands, products, or service names are or may be trademarks or service marks of, and are used to identify, products or services of their respective owners. Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in this document may require an export license from the United States government. Revision 0312
Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition
BCEFE in a Nutshell Second Edition
Objective: The BCEFE Nutshell guide is designed to help you prepare for the BCEFE Certification, exam number 150-620. Audience: The BCEFE Nutshell self-study guide is intended for those who have successfully completed the CEF 200 Certified Ethenet Fabric Engineer Training course, and who wish to undertake self-study or review activities before taking the actual BCEFE exam. The BCEFE guide is not intended as a substitute for classroom training or hands-on time with Brocade products. How to make the most of the BCEFE guide: The BCEFE guide summarizes the key topics on the BCEFE exam for you in an easy to use format. It is organized closely around the exam objectives. We suggest this guide be used in conjunction with our free online knowledge assessment test. To benefit from the BCEFE guide, we strongly recommend you have successfully completed the CEF 200 Certified Ethenet Fabric Engineer Training course. We hope you find this useful in your journey towards BCEFE Certification, and we welcome your feedback by sending an email to jcannata@brocade.com.
Joe Cannata Certification Manager
2012 Brocade Communications
Brocade Certified Ethernet Fabric Engineer in a Nutshell First Edition
ii
Table of Contents
Brocade VCS Fabric Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Brocade VCS Distributed Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Distributed Fabric Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Masterless control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Automatic distribution of zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 VCS Fabric: Use Case Aggregation Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 VCS Fabric: Use Case Combined Access/Aggregation Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Hardware Products and Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Brocade VDX Product Comparisons and Positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Brocade VDX Switch Protocol and Fabric Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Brocade VDX 6720 Data Center Switches Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Brocade VDX 6720-24 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Brocade VDX 6720-60 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Brocade VDX 6710-54 Data Center Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Brocade VDX 6710-54 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Brocade VDX 6730-32 Data Center Switch Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Brocade VDX 6730-32 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Brocade VDX 6730-76 Data Center Switch Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Brocade VDX 6730-76 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Interchangeable Field Replaceable Units (FRUs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Airflow Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Long Distance ISL Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Brocade VDX Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 VCS Fabric License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 FCoE license: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Ports on Demand (POD) license: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Brocade POD Licenses: Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 VCS Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 TRILL Defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 TRILL Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Hop Count . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 VCS Fabric Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Automatic Layer 2 Adjacency Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 L2 Forwarding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 FSPF Protocol for ECMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 VCS Layer 2 ECMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 VCS Fabric L2 Multi-Destination Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Multi-Destination Frame Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Distributed MAC Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Ethernet Name Server (eNS Lookup Table) MAC Addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Example Learning Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 VCS MAC Aging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 VCS MAC Moving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Traffic Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 VCS Known Unicast Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 VCS Multicast Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 IGMP Snooping in a VCS Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
iii
VCS Fabric Edge Loop Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 How Edge Loop Detection Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 VCS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Verifying RBridge ID and VCS ID Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Fabric ILS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Verifying VCS Fabric ISL Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Verifying VCS Fabric ISL Formation (cont.) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Verifying VCS Fabric MAC Forwarding Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 NOS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 vLAG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 vLAG Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 vLAG Example Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 vLAG Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 vLAG Optimized Multi-pathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Data Center Bridging (DCB) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 DCB Enhancements to Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 DCBX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 ETS Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 ETS and Class of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Strict Priority Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 FCoE and iSCSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 FCoE Co-Existence with LAG/vLAG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Configuring FCoE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Configuring FCoE and LLDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Enable and Disable FCoE Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 FCoE show Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 iSCI TLV Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Zone Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 FC Bridging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 FC-FC Routing to Bridge FCoE and FC Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 FC to FC Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Brocade VDX Switch TRILL Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Edge to BB Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 AMPP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 VM Mobility Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Solution Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Port Profile Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 vCenter Integration Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 NOS vCenter Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 General Ingress Traffic Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Management and Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 sFlow Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 sFlow Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 sFlow Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 sFow Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 sFlow Global Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Switched Port Analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 SPAN Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Collecting Support Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
iv
Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Login Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 TACACS+ Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 RBAC Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 SCC Policy Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 ACL Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 VDX Additional Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Port Configuration Difference for Standalone Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Trunk Mode Native VLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 VDX Hardware and Ethernet Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 Virtual IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 Taking the Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
vi
List of Figures
VCS Fabric Use Case: Aggregation Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 VCS Fabric Use Case: Combined Access/Aggregation Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Brocades VDX 6720 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Brocade VDX 6720-24 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Brocade VDX 6720-24 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Brocade VDX 6720-60 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Brocade VDX 6720-60 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Brocade VDX 6710-54 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Brocade VDX 6710-54 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Brocade VDX 6710-54 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Brocade VDX 6730-32 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Brocade VDX 6730-32 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Brocade VDX 6730-32 Ports Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Brocade VDX 6730-76 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Brocade VDX 6730-76 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Brocade VDX 6730-76 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Airflow Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 TRILL Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Layer 2 Adjacency Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 FSPF for ECMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 ECMP Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 VCS Fabric L2 Multi-Destination Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Multi-Destination Frame Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Distributed MAC Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 eNS Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 MAC Aging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 eNS MAC Aging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 MAC Moving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Updated MAC Moving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Unicast Ethernet Frame Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Unicast Ethernet Frame with TRILL Header Added . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 TRILL Ethernet Frame: Link Transport Header Added . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 TRILL Ethernet Frame: Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Ethernet Frame: End of Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 VCS Multicast Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 IGMP Snooping in a VCS Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 VCS Fabric Edge Loop Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 show VCS command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 show vcs detail command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 show fabric all command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 show fabric islports command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 show-mac-address-table command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 vLAGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 vLAGs with MCTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 vLAG Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 vLAG Optimized Multi-pathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 ETS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 ETS and Class of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
vii
FCoE Co-Existence with LAG/vLAG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 show fcoe interface brief command example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Brocade VCS Fabric to Brocade FC SAN Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 FC to FC Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Brocade VDX Switch Trill Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Edge to BB Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 VM Migration Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Port Profiles: Distributed Network Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Port Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 NOS and vCenter Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Ingress Traffic Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 sFlow Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 SPAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
viii
Brocade VCS Fabric Technology

Brocade VCS Distributed Intelligence
Distributed Fabric Services Self-forming fabric (with minimum configuration) Information shared across all fabric members Fabric aware of all connected devices
Masterless control Switch or link failure does not require full fabric reconvergence Shared port profile information Automatic Migration of Port Profiles (AMPP) Enables seamless virtual server migration
Automatic distribution of zoning Zoning changes are automatically distributed through out the fabric Brocade VCS Fabric technology supports a fully distributed control plane. That means every switch is aware of the entire network topology. You can configure it in a ring, a mesh, or a tree. You can configure enough links to make it entirely non-blocking or configure the topology to be over subscribed at whatever level you choose. Every switch knows about every attached device and Virtual Machine (VM). When a VM is attached to the network, the network configuration is automatically bound to the VM and then distributed throughout the Brocade VCS Fabric cloud. If the VM moves, the destination port already knows what to do and the VM is automatically reconnected to all of its old resources. The network configuration automatically migrates with the VM. VCS Fabric: Use Case Aggregation Layer Low cost, highly flexible logical chassis at aggregation layer. See Figure1. Building block scalability Per port price of a ToR switch Availability, reliability, manageability of a chassis Flexible subscription ratios Ideal aggregator for 1 Gbps ToR switches Optimized multi-path network
No single point failure No passive connections STP not necessary
Figure 1: VCS Fabric Use Case: Aggregation Layer
In this use case, Brocade VCS Fabric technology is used as an aggregation technology for 1 GbE server connections. This design gives the architect a building-block approach to network scalability, leveraging lowercost switches at the aggregation layer. This design approach provides the ability to scale by adding another Brocade VDX switch into the fabric. Stackable 1 GbE switches are used at ToR, and those switches are connected into an Ethernet fabric. The Ethernet fabric is very flexible and can be scaled like building blocks.
Subscription ratios can be anything, including 1:1 through the fabric. This use case example shows a threeswitch stack configuration at the ToR and two 10 GbE links per switch for a total of 6 links to the Brocade VCS Fabric technology aggregation layer through a single LAG. This building block design enables you to pay as you grow. To increase port count, simply add a Brocade VDX switch non-disruptively into the fabric. Since the Brocade VCS fabric looks and acts like a single logical entity, minimal management is required moving forward. This is an advantage of using Brocade VCS Fabric in the Access/Aggregation Layers. VCS Fabric: Use Case Combined Access/Aggregation Layers Flatter, simpler network design. See Figure2. - Logical two-tier architecture - VCS fabrics at the edge Greater Layer 2 scalability/flexibility - Increased sphere of VM mobility - Seamless network expansion Optimized multi-path network - All paths are active - No single point failure - STP not necessary Convergence ready - End-to-end enhanced Ethernet (DCB) - Multi-hop FCoE support - iSCSI DCB support
Figure 2: VCS Fabric Use Case: Combined Access/Aggregation Layers
Storage is now introduced to the Ethernet fabric. This use case illustrates end-to-end native multi-hop FCoE and iSCSI DCB connectivity. Also, instead of any core, it is a Brocade MLX with DCB capabilities. This allows for lossless fabric-to-fabric communication. This use case illustrates two ways the fabric can be configured. In this diagram, a ToR Mesh architecture is used. The benefit is that a true flat network edge, where the switches are connected to its peers. The challenge of this design is to maintain common subscription ratios between each server (between racks). This topology is a single stage design.
Hardware Products and Features

Brocade VDX Product Comparisons and Positioning
Table 1: Brocade VDX Product Comparisons and Positioning
Brocade VDX 6710 Physical Description Brocade VDX 6720 Brocade VDX 6730
1U 6 x 10 GbE SFP+ and 48 x 1 GbE GE copper
2U 24 x 10 GbE SFP+ ports 60 x 10 GbE SFP+ ports
2U 24 x 10 GbE SFP+ and 8 x 8 Gbps FC ports 60 x 10 GbE SFP+ ports and 16 x 8 Gbps FC ports POD License Brocade VCS Fabric License FCoE License
Licenses
Brocade VCS Fabric License
POD License Brocade VCS Fabric License FCoE License
Positioning
High Performance, low latency, enabled 1 Gbps switch providing costeffective connectivity to 1G servers
High performance, low latency, enabled 10 GbE switch targeted at virtual data centers. Enables endto-end LAN/SAN convergence solutions with VCS fabric
High Performance, low latency enabled convergence switch with FC connectivity
Brocade VDX Switch Protocol and Fabric Support

Table 2: Brocade VDX Switch Protocol and Fabric Support
Product Brocade VDX 6710 Brocade VDX 6720 Brocade VDX 6730 iSCSI Yes Yes Yes
1
Ethernet Yes Yes Yes
FCoE No Yes Yes

2
FC No No Yes
4
VCS Fabric Yes 3 Yes Yes
The Brocade VDX 6710 and 6730 switches support convergence through iSCSI, FCoE, and FC protocol support and also are part of a VCS fabric. The VDX 6710 is a Ethernet switch that can be part of a VCS fabric. Note 1: The 1GbE ports do not support DCB so the iSCSI TLV is not supported however iSCSI traffic can still go across these ports. iSCSI TLV is supported on the 10 GbE ports.
Note 2: Cannot attach an FCoE device to a port on this switch however FCoE traffic can flow through the 10 G (Trill) ports as long as the destination is not on that switch. As the switch would look at the destination RB and modify the outer header with next hop information so the switch would never look at the type of traffic in that frame. Note 3: The 1 GbE ports can not be used as ISL connections to other switches in the fabric. Note 4: The FC ports can only be used as E_Port (Brocade VDX switch) to EX_Port (SAN router) connection. Cannot directly attach a FC end device to any of the FC ports on the Brocade VDX 6730.
Brocade VDX 6720 Data Center Switches Overview

Built for the Virtual Data Center. See Figure3. - Uses Brocade fabric switching eAnvil2 ASIC - Supports Brocade Network Operating System (NOS) including VCS technology Performance and Density - 24- and 60-port models (VDX 6720-24 and VDX 6720-60) - Ports on Demand (POD) enables 24 to 60 port configurations - 600 ns port-to-port latency Configuration Flexibility - 1 Gbps or 10 Gbps supported on every port - Twinax, direct-attached optical, and SFP optical connectivity options - Front-to-back or back-to-front airflow Enables Network Convergence - Complete FCoE support including multi-hop (license required) - iSCSI Data Center Bridging support (DCB)
Figure 3: Brocades VDX 6720
Brocade VDX 6720-24 Data Flow Brocade VDX 6720 supports port-to-port communication through a single eAnvil2 ASIC: The control plane packets are sent to the CPU for routing processing are represented by the wide green line Ethernet traffic is indicated with yellow lines. As shown Figure4, VDX 6720-24 platform contains one eAnvil2 ASIC. Each eAnvil2 ASIC supports a Routing Engine (RTE) and each RTE supports 24 ports, 1 or 10 GbE. Every packet coming in through the external SFP+ interfaces destined to external ports are switched in a single stage within the eAnvil2 ASIC. The data path is also applicable to packets originated by the CPU which need to go out of the network ports. Switching bandwidth from switch backplane (data rate at full duplex): 6720-24: 480 Gbps Forwarding bandwidth from ports (data rate at full duplex): 6720-24: 358 Mpps (Million packets per second: used in measuring forwarding rate and forwarding bandwidth) The latency for port-to-port communication is 600ns Port groups are shown in Figure5.
Figure 4: Brocade VDX 6720-24 Data Flow
Figure 5: Brocade VDX 6720-24 Port Groups
Brocade VDX 6720-60 Data Flow

The Brocade VDX 6720-60 platform contains nine eAnvil2 ASICs: Six front end eAnvil2 ASICs Each provides 10 external SFP+ interfaces Three back end eAnvil2 ASICs Interconnectivity between the front end ASICs is achieved via the back end eAnvil2 ASICs. The six front facing eAnvil2 ASICs support local switching. For example, if port 1 needs to send the data to port 2, the packet can be switched locally within a single eAnvil2 ASIC and does not require inter-ASIC switching Unlike the VDX 6720-24 design, the incoming packets on SFP+ ports destined to an external port now go through three different eAnvil2 ASICs if the source and destination port are on different eAnvil2 ASICs.
Switching bandwidth from switch backplane (data rate at full duplex): 6720-60: 1200 Gbps Forwarding bandwidth from ports (data rate at full duplex):6720-60: 896 Mpps Port groups are shown in Figure7. Based on which ASIC the source port and the destination port are using, the data may take a single-hop (local switching) or two-hops (inter-ASIC switching) as shown below. Each line represents 4 x 10 Gbps. See Figure6.
10
Brocade VDX 6710-54 Data Center Switch

The Brocade VDX 6710-54 is a low-cost Ethernet switch that is VCS capable. Uses Brocade fabric switching eAnvil2 ASIC and supports Brocade Network Operating System (NOS) including Brocade VCS Fabric technology. See Figure8. 54 total ports 6 x 10 GbE DCB capable optical ports. The 10 GbE ports are designed to up used as uplink ports to the VCS fabric however do support the direct attachment of 10 GbE devices. FCoE direct attached devices are not supported on this switch. 48 x 1 GbE copper ports Configuration flexibility: Front-to-back or back-to-front airflow
11
Figure 8: Brocade VDX 6710-54 Brocade VDX 6710-54 Data Flow The Brocade VDX 6710-54 platform contains three eAnvil2 ASICs. eAnvil2 1: Front end ports are 14 x 1 GbE ports and 6 x 10 GbE ports. It also has 4 x 10 GbE back end connections to the eAnvil2 2 ASIC. eAnvil2 2: Contains 13 x 1 GbE front end ports and acts as a back end core ASIC for frame routing between the ASICs. There are 2 x 10 GbE connections from the front end to the back end of the ASIC for routing to the other two ASICs. eAnvil2 3: Contains 21 x 1 GbE front end ports and 3 x 10 GbE back end ports Port groups are shown in Figure10. Over subscription: There is no over subscription. From the Main CPU there is a connection to each of the three ASICs. Shown is only the connection to one of the eAnvil2 ASICs. See Figure9.
12
13
Brocade VDX 6730-32 Data Center Switch Overview

Uses Brocade fabric switching eAnvil2 ASIC. See Figure11. Supports Brocade Network Operating System (NOS) including Brocade VCS Fabric technology Configuration flexibility: Ports on Demand (POD) enables ports to be added. Default is 16 ports enabled with a single 8 POD license available to support a total of 32 ports 32 total ports 8 x 8 Gbps FC ports 24 x 10 GbE DCB capable ports. Twinax and SFP optical connectivity options Front-to-back or back-to-front airflow Supports Network Convergence: Complete FCoE support including multi-hop (license required) and iSCSI Data Center Bridging support (DCB).
Figure 11: Brocade VDX 6730-32
Brocade VDX 6730-32 Data Flow As shown above, the Brocade VDX 6730-32 platform contains one eAnvil2 and one Condor2 ASIC. The eAnvil2 ASIC support 2 RTE (Routing Engine) and each RTE supports 12, 10/1 Gbps Ethernet Ports. So every packet coming in through the external SFP+ interfaces destined to external ports are switched in a single stage within the eAnvil2 ASIC. See Figure12. Port groups are shown in Figure13. There is no over subscription for network traffic. FC Traffic over subscription worst case would be 240:64 (15:4) 240: 24 x 10 GbE FCoE devices
14
64: 8 x 8 Gbps FC traffic between the eAnvil2 ASIC and the Condor 2
15
Figure 13: Brocade VDX 6730-32 Ports Groups
16
Brocade VDX 6730-76 Data Center Switch Overview

Uses Brocade fabric switching eAnvil2 ASIC. See Figure14. Supports Brocade Network Operating System (NOS) including Brocade VCS Fabric technology Configuration flexibility: Ports on Demand (POD) enables ports to be added. Default is 40 ports enabled with two 10 POD licenses available to support a total of 60 GE ports 76 total ports 16 x 8 Gbps FC ports 60 x 10 GbE DCB capable ports: Twinax and SFP optical connectivity options Supports Network Convergence: Complete FCoE support including multi-hop (license required) and iSCSI Data Center Bridging support (DCB)
Figure 14: Brocade VDX 6730-76
Brocade VDX 6730-76 Data Flow As shown above, the forwarding plane on Brocade VDX6730-76 platform contains nine eAnvil2 ASICs and one Condor2 ASIC. There are six eAnvil2 ASICs, each providing 10 external SFP+ interfaces, which are connected to three additional eAnvil2 ASICs providing the interconnectivity. The six front facing eAnvil2 ASICs support local switching. i.e if port-1 needs to send the data to port-2, the packet can be switched locally within a single eAnvil2 ASIC. However unlike the VDX6730-32 design, the incoming packets on SFP+ ports destined to an external port can now go through three different eAnvil2 ASICs if the source and destination port are on different eAnvil2 ASICs. See Figure15. The Condor2 ASIC provides Channel switching functionality. Each of the six eAnvil2 ASICs are connected to the Condor2 ASIC using a 4 link trunk. The Condor2 ASIC provides 16 external 8G Fibre Channel interfaces. Over subscription: There is no over subscription for traditional network traffic (non-FCoE) as each front end ASIC has 10 x 10 GbE front end ports and 4x10 to each of the 3 back end ASICs to the maximum front end bandwidth would be 10 x 10 GbE = 100 GbE. The back end connections are 4 links x 10 GbE x 3 (ASICs) = 120 GbE bandwidth. FC over subscription: The FC connections the front end eAnvil2 ASICs and the Condor2 is 4 x 8 so the worst case would be to have 10 FCoE devices on one front end ASIC (example ports 1-10). In that case the worst case FCoE traffic would be 100 GbE to 32 FC (bandwidth connection from eAnvil2) to Condor2) 100:32 (25:8). FCoE best practice: Unless you know the FCoE usage limit the FCoE connections 3 per ASIC. If this is done there would be no over subscriptions on the internal connections. Of course there would still be worst case
17
over subscriptions 3 (FCoE connections) x 10 GbE x 6 ASICs = 180 GbE to 16 front end links x 8 Gbps = 128 Gbps (180:128 or 45;32). To insure no over subscription limit the FCoE device to 12 and put no more then 3 per eAnvil2 ASIC. Port groups are shown in Figure16.
18
Interchangeable Field Replaceable Units (FRUs) The VDX 6730-76 power supply can be used with the VDX 6720-60 switch. You can interchange the power supply FRU but make sure the power supply FRU uses the correct airflow. The airflow options are described in more detail later in this module. The integrated power supply/fans for the VDX 6710-54 and 6730-32 share the same part number. You can interchange the integrated power supply/fans but make sure the power supply/fan uses the correct airflow. The airflow options are described in more detail later in this module. Also, the integrated power supply/fan FRU can also be used with the VDX 6720-24. On the non-port side of the switch enclosure, each power supply/fan FRU has a green/amber LED to indicate the status of the FRU. A steady green light indicates that the FRU is on and operating normally. If there is an amber light, the power is faulty.
19
Airflow Options Some high-density server racks have specific cooling requirements. The Brocade VDX Data Center switches support the following configurations:
Front-to-back or back-to-front airflow Airflow direction determined by integrated power/fan unit, power supply, or fan installed
Note: Must order the FRU with correct airflow direction
Figure 17: Airflow Labels
When ordering FRUs, verify that the part number supports the correct airflow. You must replace a failed unit with the same type of unit. This applies to both power supplies and fan assemblies. A new FRU must have the same part number (P/N) as the FRU being replaced. The manufacturing P/N is located on the top of the FRU. The P/N ends in either -F (front-to-back airflow) or -R (back/rear-to-front airflow). You must use a replacement FRU that the same airflow designator with the part number. Additionally, you can use external labels as a guide. Some FRUs are labeled with an airflow symbol on the faceplate to indicate whether the assembly takes in or exhausts air. The symbol also appears on the top of the assembly.
20
Long Distance ISL Support

With NOS v2.1.1 or higher, the maximum ISL distance is 10 km Supported on the Brocade VDX 6720-60 and 6730-76 switches All other Brocade switches support a maximum distance of 200 m Use the long-distance-isl command to specify one of these distances: 200 m (default) 2000 m 5000 m 10000 m Here is an example:
VDX11(conf-if-te-11/0/14)# long-distance-isl ?
Possible completions:
2000 2000 meter distance link (Warning: It May disable other ISLs in the port group) 5000 5000 meter distance link (Warning: It May disable other ISLs in the port group) 10000 10,000 meter distance link (Warning: It May disable other ISLs in the port group)
Additional notes: For a 10 km ISL link, no other ISL links are allowed on the same eAnvil2 ASIC. For 2 km and 5 km ISL links, another short distance ISL link can be configured. A maximum of 3 PFCs (per priority flow control) can be supported on a long distance ISL link. Use the no long-distance-isl command to revert to the default value which is 200 m. For more information refer to the Brocade Network OS Administrators Guide Supporting Network OS v2.1.1, 53-1002491.
21
Brocade VDX Licenses

VCS Fabric License If the Brocade VCS Fabric does not exceed two Brocade VDX switches, no VCS Fabric license is required A two switch Brocade VCS Fabric license is built into NOS If there are more than two nodes in the fabric, each switch requires a VCS Fabric license FCoE license: Required to support FCoE on Brocade VDX 6720 and 6730 switches Required to activate FC ports on Brocade VDX 6730-32 and 6730-76 switches. An FCoE license enables FCoE on the Brocade VDX platforms. Since FCoE requires a VCS fabric, a VCS Fabric license is a prerequisite for enabling FCoE in a fabric with more than two switches. In addition, a separate FCoE license is required to enable FCoE on a VCS edge port. Without an FCoE license, FCoE logins are not permitted, and all FCoE commands with the exception of a few commands return an error of No FCoE license present when executed. See release notes for more information on FC port activation and the FCoE license. Ports on Demand (POD) license: Brocade VDX 6720-24 and 6730-32: One 8-port POD license Base switch has 16 ports enabled Brocade VDX 6720-60 and 6730-76: Two 10-port POD licenses Base switch has 40 ports enabled A Dynamic POD license allows you to instantly scale the fabric by provisioning additional ports on the Brocade VDX platforms. Licenses are assigned dynamically from a pool of resources based on auto-detection of active links.
VDX# show license id rbridge-id License ID ======================================== 2 10:00:00:05:1E:00:4C:80
Brocade POD Licenses: Additional Information Ports use a dynamic POD system, each port can be associated with one of three port sets: Base Port Set: Ports that can be enabled without any POD license Single POD License Port Set: Ports that are assigned first and associated with the existence of a single POD license Double POD License Port Set: Ports that are assigned after the single POD port set is full and are associated with the double POD license If no POD license is installed no other ports can be used unless a port is released
22
Example: On a Brocade VDX6720-24 the first 16 ports that are used (regardless of which physical ports they are) will be assigned to the base port set The Brocade VDX6720-24 only has one POD license. Use the dpod slot/port release command to release a port. Use the dpod slot/port reserve command to reserve a port for future use. Use the show dpod command to view port reservations.
VDX3# sh dpod rbridge-id: 3 24 ports are available in this switch 2 POD licenses are installed Dynamic POD method is in use 24 port assignments are provisioned for use in this switch: 16 port assignments are provisioned by the base switch license 8 port assignments are provisioned by the first POD license 0 port assignments are provisioned by the second POD license 2 ports are assigned to installed licenses: 2 ports are assigned to the base switch license 0 ports are assigned to the first POD license Ports assigned to the base switch license: Te 3/0/7, Te 3/0/8 Ports assigned to the first POD license: None Ports assigned to the second POD license: None Ports not assigned to a license: Te 3/0/1, Te 3/0/2, Te 3/0/3, Te 3/0/4, Te 3/0/5, Te 3/0/6, Te 3/0/9, Te 3/0/10, Te 3/0/ 11, Te 3/0/12, Te 3/0/13, Te 3/0/14, Te 3/0/15, Te 3/0/16, Te 3/0/17, Te 3/0/18, Te 3/0/19, Te 3/0/20, Te 3/0/21, Te 3/0/22, Te 3/0/23, Te 3/0/24 22 license reservations are still available for use by unassigned ports
23
VCS Fabric
Brocade VCS Fabric technology leverages proven Fibre Channel fabric protocols to build a TRILL-based fabric.
TRILL Defined
Transparent Interconnection of Lots of Links (TRILL) provides a solution for shortest path frame routing for multihop Layer 2 Ethernet that:
Supports arbitrary topologies (ring, mesh, star) Uses link-state routing protocols - The link-state protocol is performed by every switching node in the network. The basic concept of linkstate routing is that every node constructs a map of the connectivity to the network, in the form of a graph, showing which nodes are connected to which other nodes. Each node then independently calculates the next best logical path from it to every possible destination in the network. The collection of best paths will then form the node's routing table.
Brocade uses Fabric Shortest Path First (FSPF) as the routing protocol TRILL is currently a proposed Internet Engineering Task Force (IETF) standard and is an Equal-Cost Multi-Path (ECMP)-capable protocol. Equal-cost multi-path routing (ECMP) is a routing strategy where next-hop packet forwarding to a single destination can occur over multiple "best paths" which tie for top place in routing metric calculations. Multipath routing can be used in conjunction with most routing protocols, since it is a per-hop decision that is limited to a single router. It potentially offers substantial increases in bandwidth by load-balancing traffic over multiple paths
TRILL uses a link state-based control plane to form loop-free optimized paths between a source and destination. TRILL provides the following features:
Minimal configuration required Load balancing among multiple paths Forwarding loop mitigation without the need for STP Support of multiple points of attachment to the TRILL network. In STP, a single node with multiple attachments to a single spanning tree segment always receives and sends traffic over only one of the those attachment points. TRILL must manage all traffic, including multicast and broadcast traffic, so as not to create traffic loops involving Ethernet segments with multiple TRILL attachment points. This includes multiple attachments to a single TRILL node and attachments to multiple TRILL nodes. Support for multiple attachments can improve support for forms of mobility that induce topology changes
Support for broadcast and multicast Behaves as a normal bridge to devices outside the TRILL network
TRILL Frames TRILL encapsulates an Ethernet frame to specify information to route the frame through the network (Standard frames as well as 2.5 k Mini Jumbo frames are also supported). See Figure18.
24
Link transport header: A header that is added and removed at each hop and is used to specify the next hop RBridge and the transmitting RBridge This is also known as the outer MAC header TRILL header: Specifies nickname values of the egress and ingress RBridges, unless the frame is multidestined, in which case the egress nick.name specifies the root of the distribution tree on which the frame is being sent. The egress nickname is Equivalent to the RBridge ID.
Figure 18: TRILL Frame
Hop Count The Hop Count field is a 6-bit unsigned integer. An RBridge drops frames received with a hop count of zero, otherwise it decrements the hop count. (This behavior is different from IPv4 and IPv6 in order to support the later addition of a trace route-like facility that would be able to get a hop count exceeded from an egress RBridge.) For known unicast frames, the ingress RBridge should set the hop count in excess of the number of RBridge hops it expects to the egress RBridge to allow for alternate routing later in the path. For multi-destination frames, the hop count should be set by the ingress RBridge to at least the expected number of hops to the most distant RBridge. To accomplish this, RBridge RBn calculates, for each branch from RBn of the specified distribution tree rooted at RBi, the maximum number of hops in that branch. Multi-destination frames are of particular danger because a loop involving one or more distribution tree forks could result in the rapid generation of multiple copies of the frame, even with the normal TTL mechanism. It is for this reason that multi-destination frames are subject to a stringent Reverse Path Forwarding Check and other checks. As an optional additional traffic control measure, when forwarding a multi-destination frame onto a distribution tree branch, transit RBridge RBn may decrease the hop count by more than 1, unless decreasing the hop count by more than 1 would result in a hop count insufficient to reach all destinations in that branch of the tree rooted at RBi. Using a hop count close or equal to the minimum needed on multi-destination frames provides additional protection against problems with temporary loops when forwarding. Although the RBridge MAY decrease the hop count of multi-destination frames by more than 1, under the circumstances described above, the RBridge forwarding a frame must decrease the hop count by at least 1, and discards the frame if it cannot do so because the hop count is 0. The option to decrease the hop count by more than 1 under the circumstances described above applies only to multi-destination frames, not to known unicast frames.
25
VCS Fabric Formation

Brocade VCS Fabric technology leverages proven Fibre Channel fabric protocols to build a TRILL-based fabric. The main functions of the fabric formation protocols are:
Confirming that each switch in the VCS fabric is assigned the same VCS ID Confirming that each switch in the VCS fabric is assigned a unique RBridge ID Create a network topology database using a link state routing protocol (FSPF) Compute a broadcast tree to distribute fabric broadcast and multicast traffic
Before connecting VDX switches, the following should be configured:
Set the VCS Fabric ID Set the RBridge ID Enable VCS Fabric mode
Automatic Layer 2 Adjacency Formation Brocade VCS Fabric forms adjacencies with its directly connected Brocade VCS Fabric-enabled neighboring switches. See Figure19. These adjacencies are called Inter Switch Links (ISLs) The ISLs can be separate links or Brocade proprietary VCS Fabric trunks
Figure 19: Layer 2 Adjacency Formation
26
RBridge ID Conflicts If there is a conflict for an RBridge ID, one of the offending switches must have their RBridge ID changed Valid RBridge IDs are 1-239 An RBridge ID change requires a reboot of the switch.The switch needs to be rebooted because an RBridge ID change requires a reset to all configuration databases to accommodate the new RBridge ID VCS Edge Port Configuration VCS fabric edge ports are switch ports used to connect to external devices including end-stations or non-VCS Fabric mode switches or routers. Standard Layer 2 configuration options are available (i.e. VLANs and LAGs) STP is not configurable on edge ports. STP is not configurable on edge ports, but VCS will pass BPDU traffic through the fabric. STP sees the fabric as a wire. When an edge port converts to an ISL port, the edge port configuration is ignored. Similarly, when an ISL port converts to an edge port, all the user provisioned edge port configuration for that port is activated.
27
L2 Forwarding
FSPF Protocol for ECMP

Brocade VCS Fabrics uses the FSPF routing protocol to distribute link-state information of all ISLs. FSPF is a Link State Path Selection protocol, similar to OSPF, which is an Interior Gateway Protocol (IGP) widely used in IP networks. FSPF keeps track of the state of the links on all switches in the Fabric. It also associates a cost with each link. The protocol computes paths from a switch to all the other switches in the fabric, by adding the cost of all the links traversed by the path, and choosing the path that minimizes the cost. See Figure20. FSPF is similar to Layer 3 routing protocols like OSPF. Although it has roots from OSPF, FSPF only defines and implements point-to-point links. In other words, there is no concept of a designated router (DR) and a backup designated router (BDR), areas or summarization, or anything similar like that being managed in FSPF. FSPF forms a single adjacency per fabric trunk.
Figure 20: FSPF for ECMP
Table 3: FSPF Key to Figure20

From RB3 Destination RB RB1 RB2 RB4 RB5 RB6 Outgoing Interface te 3/0/1, te 3/0/3 te 3/0/1, te 3/0/3 te 3/0/1, te 3/0/3 te 3/0/1 te 3/0/3
28
VCS Layer 2 ECMP With NOS v2.0.0a, up to 8 ECMP paths are supported per switch. When multiple ECMP paths exist, the traffic is load balanced across all available equal cost paths based on a hash from fields in the frames. The hash algorithm can be changed using the fabric ecmp load-balance command. For more information see the.Network OS Administrators Guide Supporting Network OS v2.1.1 manual. See Figure21.
IP: MAC DA, MAC SA, VLAN, IP DA, IP SA, TCP/UDP ports FCoE: Input port, MAC DA, MAC SA, VLAN, D_ID, S_ID, OX_ID Other: MAC DA, MAC SA, VLAN
Figure 21: ECMP Paths
VCS Fabric L2 Multi-Destination Tree VCS Fabric technology uses FSPF to calculate a loop-free multi-destination tree root switch. The multidestination tree is calculated after the unicast routes are computed. The following rules determine the multidestination root RBridge: RBridge ID with the highest priority. The default multicast RBridge priority is zero. The default can be changed using the fabric route mcast RBridge ID <RB-ID> priority <priority> command. The priority range is 0 through 255. Lowest RBridge ID When multiple links are available between two nodes in the multicast tree The one with the highest BW at that instant is selected. Like if there is a ISL with 40G and a ISL with 20G then the 40G is chosen. If there are more than one equal bandwidth ISLs, then the port on the lowest switch WWN and lowest port will be chosen. The following rules determine which down stream links will be selected. Highest bandwidth Lowest WWN - lowest port on that switch In Figure22, RB5 is not the lowest RBridge number but it has been assigned the highest priority
29
Figure 22: VCS Fabric L2 Multi-Destination Tree
Multi-Destination Frame Distribution Assume multi-destination traffic is received on the ingress RB_ID=2 (RB2). See Figure23. RB2 delivers traffic to all local edge ports (RB6 and RB5) that participate in the same VLAN RB2 encapsulates the frame using: An outer MAC SA of the transmitting port MAC An outer MAC DA of All RBs RB2 sends this encapsulated frame to its upstream and downstream paths in the tree. Multicast Root (RB5) distributes the traffic to other branches of tree (RB1, RB3, and RB4), but not on the branch it is received (RB2).
30
Figure 23: Multi-Destination Frame Distribution
Distributed MAC Learning

Ethernet Name Server (eNS Lookup Table) MAC Addresses When the destination MAC address is not in the lookup table the frame is flooded on all ports in the same VLAN, except the ingress port. When the destination MAC address is present in the lookup table the frame is switched only to the correct egress port. If the egress port is the same as the ingress port, the frame is dropped. MAC addresses in the eNS lookup table are removed when one of the following occur:
The MAC address times out A device is moved to a new location

When a device is moved, the ingress frame from the new port causes the old lookup table entry to be discarded and the new entry to be inserted into the lookup table. Frame forwarding remains unicast to the new port.
The lookup table is full

When the lookup table is full, new entries replace the oldest MAC addresses after the oldest MAC addresses reach a certain age and time out. MAC addresses that still have traffic running are not timed out. New entries start replacing older entries when the lookup table reaches 90 percent of its 32K capacity.
31
Example Learning Process The frame is forwarded into the fabric with TRILL encapsulation based on whether the Destination Address (DA) in the frame is known or unknown. See Figure24.
If the DA is known to exist on a specific RBridge, then it is sent to that specific destination RBridge. If the DA is unknown, then the frames go out on the multicast tree that covers all RBridges in the fabric.
Figure 24: Distributed MAC Learning Note: RB1 is the multicast root bridge.
The first time an unknown DA is encountered, flooding occurs. SeeFigure25.
When the MAC address is learned anywhere in the Ethernet fabric, the eNS MAC distribution service
distributes the learned MAC, interface, VLAN and RBridge entry using the control plane
The entry is distributed to every VCS member switch to complete the missing information that is not
provided by TRILL
This method minimizes flooding
Figure 25: eNS Distribution
VCS MAC Aging MAC address aging logic on each switch is identical to how aging is performed on a standard IEEE 802.1Q switch. See Figure26.
Addresses that are learned on local edge port interfaces are aged by the aging function local to the
switch.
32
In VCS mode, the local aging function informs eNS of an entry that is aged out after it ages it out locally. Similarly, addresses are aged out using eNS.
Figure 26: MAC Aging
eNS then informs all the other switches in the fabric to have the forwarding entry removed. The default aging time is 300 seconds. See Figure27. Only the physical switch that the MAC resides on can age a MAC.
Figure 27: eNS MAC Aging
VCS MAC Moving When MAC A starts sending frames on RB3, RB3 learns about MAC A locally and immediately sees a conflict. See Figure28.
33
RB3 updates its local table.
Figure 28: MAC Moving
MAC A now resides on RB3, so RB3 has the responsibility for updating the fabric. See Figure29. RB3 then sends the update to eNS and eNS distributes the change to the other VCS member switches
Figure 29: Updated MAC Moving
34
Data Path
Traffic Types The data path through the fabric changes depending on the type of traffic. Traffic is classified into two distinct types:
Known unicast (destination MAC is known) Broadcast, unknown unicast, or multicast

VCS Known Unicast Data Path For a known unicast frame, no flooding occurs when the one of the following occurs:
MAC has been previously learned RBridges know the RB that is hosting the MAC FSPF knows the shortest path to get to the destination RBridge because the routing topology has already
been created
35
Learning of a source MAC to an ingress RBridge is done through the data plane. The ingress RBridge encapsulates the native Ethernet frame with TRILL headers until the frame reaches the destination RBridge. The outer MAC header is changed at every hop. The frame enters the fabric through the ingress RB (RB1). The source MAC (ES_A) is learned by RB1. See Figure30.
Figure 30: Unicast Ethernet Frame Example
36
The ES_B is a known MAC and RB1 knows that MAC ES_B is hosted by RB2 RB1 adds a TRILL header with a egress RB nickname of RB2. See Figure31.
Figure 31: Unicast Ethernet Frame with TRILL Header Added
37
RB1 uses the FSPF routing table to determine the path to RB2. See Figure32. RB1 adds the link transport header which specifies: - The next hop RB for the path (RB3) - Itself as the Outer MAC SA
Figure 32: TRILL Ethernet Frame: Link Transport Header Added
38
As a transit switch, RB3 removes the previous link transport header and replaces it with one that specifies
the next hop RB. See Figure33.
Figure 33: TRILL Ethernet Frame: Data Path
39
The frame arrives at the egress RB, RB2 RB2 removes the TRILL header and delivers the frame to the end station. See Figure33.
Figure 34: Ethernet Frame: End of Data Path
40
VCS Multicast Data Path Unknown MAC, broadcast, or multicast traffic enters the fabric
Traffic is flooded and sent to multicast root. See Figure35.
Figure 35: VCS Multicast Data Path
IGMP Snooping in a VCS Fabric IGMP snooping is the process of listening to Internet Group Management Protocol (IGMP) network traffic. IGMP snooping, as implied by the name, is a feature that allows a Layer 2 switch to listen in on the IGMP conversation between hosts and routers. By listening to these conversations the switch maintains a map of links that need specific IP multicast streams. Multicasts may then be sent only to the links that need them.All IGMP snooped traffic rides on the fabric multicast tree. See Figure36. All multicast traffic known and unknown is sent to all RBridges through the multicast tree All unknown multicast groups are flooded devices attached to the fabric
41
If traffic for a known multicast group does not have any receivers on that RBridge then the traffic is dropped
Figure 36: IGMP Snooping in a VCS Fabric
VCS Fabric Edge Loop Detection Edge-loop detection (ELD) detects and disables loops that would cause broadcast storms. These loops are generally caused by misconfigurations. See Figure37. ELD is configured and enabled on Brocade VCS Fabrics ELD is not supported on switches in standalone mode
42
Figure 37: VCS Fabric Edge Loop Detection
How Edge Loop Detection Works ELD works by multicasting Protocol Data Unit (PDU) packets on edge ports. If a port is blocked by STP or another L2 protocol, ELD pdus are not processed or flooded on that port. A RBridge recognizes a loop when it receives a PDU that it initiated. If an ELD pdu is received on a port/VLAN that is not configured with ELD, the pdu will be dropped. Once an RBridge recognizes that a loop exists, it shuts down a port and breaks the loop. In the case of LAG ports, ELD pdu are sent out on the active primary port of a lag group. ELD protocol will shut down all the member ports of the lag if an ELD pdu is received on any member port. In case of vLAG ports, ELD pdu are sent out on the active primary port of the vLAG. ELD protocol will shut down all the member ports of the vLAG on the RBridge receiving the pdu. If there are other member ports exist on any other RBridge, they will still be up. This way loop will be broken and connectivity will still be maintained.
43
VCS Configuration
Verifying RBridge ID and VCS ID Assignment
After assigning an RBridge ID, a VCS ID, and rebooting, you can verify the VCS Fabric configuration using the show vcs command. The coordinator is also known as the principal switch. This output in Figure38 shows the VCS_ID, total number of switches in the VCS fabric, their RBridge IDs, WWNs, Management port IP address, status and Hostname (if assigned, the default is sw0). See next slide on how to set hostname.
Figure 38: show VCS command output
The command show vcs detail provides additional details as shown in Figure39.
Figure 39: show vcs detail command output
44
Fabric ILS Configuration

No configuration needed for normal ISL operation (default configuration is already configured)
interface TenGigabitEthernet 11/0/1 fabric isl enable fabric trunk enable no shutdown
ISLs can be shutdown and have ISL and trunk functionality turned on or off
VDX11(config-if-te-11/0/1)# [no] shutdown VDX11(config-if-te-11/0/1)# [no] fabric isl enable VDX11(config-if-te-11/0/1)# [no] fabric trunk enable
No edge port configuration is allowed on an active ISL

VDX11(config-if-te-11/0/1)# switchport % Interface Port Role is ISL, nothing to be done
This command controls whether an ISL is formed between two cluster members. With the default setting of ISL discovery mode set to auto, and ISL administrative mode set to enable, an ISL automatically forms between two fabric switches. Performing the fabric isl enable command on an operational ISL has no effect. However, performing the no fabric isl enable command on an interface toggles its link status, and subsequently disables ISL formation. In addition, the no fabric isl enable command triggers the switch to inform its neighbor that the local interface is ISL disabled. Upon receiving such information, a neighbor switch stops its ISL formation activity regardless of its current interface state. Note that a shutdown on an operating ISL interface not only brings down the physical link but also its FSPF adjacency. The main difference between a shutdown and the no fabric isl enable command is that the link stays up after the no fabric isl enable command has been issued, whereas the link stays down after a shutdown. It is recommended that users use the no fabric isl enable command to expedite ISL state transition as its link state stays up. The no fabric isl enable command has no effect on an interface when no fabric isl mode auto is configured. In addition, the no fabric isl enable command on a member link disassociates the link with its trunk group.
Verifying VCS Fabric ISL Formation

The show fabric all command is equivalent to the FOS fabricshow command as shown in Figure40. Use the show fabric all command to display information about the fabric. If the switch is initializing or is disabled, the message Local Switch disabled or fabric is re-building is displayed. If the fabric is re-configuring, some or all switches may not be displayed; otherwise, the following fields are displayed. See Figure40. VCS Id: VCS id of the switch Config Mode: VCS mode of the switch. For fabric cluster mode, "Local-Only" is displayed.
45
RBridge-id: The RBridge-id of the switch. WWN: The switch World Wide Name. IP Address: The switch Ethernet IP address. Name: The switch symbolic name. An arrow (>) indicates the coordinator or principal switch. An asterisk (*) indicates the switch on which the command is entered.
Figure 40: show fabric all command output
Verifying VCS Fabric ISL Formation (cont.) Use the show fabric islports command to display information for all ISL ports in the switch as show in Figure41. Trunk primary is equivalent to trunk master. This is similar to the FOS switchshow command. The command output includes the following information: Name: Switch name. Type Switch: model and revision number. State: Switch state. The valid values are Online, Offline, Testing, or Faulty. Role: Switch role. The valid values are Principal, Subordinate, or Disabled.
46
VCS Id: The range of valid values is from 1 through 8192 Config Mode: VCS mode. The valid values are Standalone/Local-Only/Distributed. RBridge-id: RBridge-id of the switch. The range of valid values is from 1 through 239. WWN: Switch world wide name (WWN). FCF MAC: Mac address Index: Port Index is a number between 0 and the maximum number of supported ports on the platform. The port index identifies the port number relative to the switch. Interface: Interface of the local RBridge in the format "local-rbridge-id/slot/port". State: Port state information:
Up If the ISL is connected and the link is up Down No ISL is connected
Operational State: Displays the operational state of the ISL
Figure 41: show fabric islports command output
47
Verifying VCS Fabric MAC Forwarding Database Use the show mac-address-table command to display a specific static or dynamic MAC address entry or all entries for a specific interface, a specific VLAN, a specific linecard, or for all interfaces and all VLANs. VlanId: Displays the VLAN. Mac-address: Displays the MAC address of the external device. Type: Displays the of MAC address, Static, Dynamic or FPMA (Fabric Provided MAC address). Ports: Displays the Physical interface that MAC address is attached too.
Note: The inactive state of the static mac means that a static entry has timed out, the MAC but because it is a static MAC it has not been removed from the table. See Figure42.
Figure 42: show-mac-address-table command output
48
NOS Configuration
NOS configuration management uses three types of configuration files: The default configuration files are provided as part of the NOS firmware package. Default configuration files: The default configuration is used, if no startup configuration is available. There are different default configuration files for standalone and VCS Fabric mode. The default configuration files are provided as part of the Network OS 2.1.0 or higher firmware package. The default configuration is applied, if no customized configuration is available. There are different default configuration files for standalone and VCS mode: defaultconfig.novcs and defaultconfig.vcs Start-up configuration file: Committed user-defined configuration that becomes effective after reboot Running configuration: Current configuration active on the switch. Whenever the configuration is changed, it is written to the running configuration The running configuration does not persist across reboots, unless it is copied to the start-up configuration To copy the running configuration to the startup configuration run the following command:
VDX# copy running-config startup-config This operation will modify your startup configuration. Do you want to continue? [y/n]:y
At the next switch reboot, the contents of the startup-config become the running-config This command can not be shortcutted however you can use tab to complete the command option. Example you can not type in: copy run start However you can type in: copy run[tab] start[tab] and the tab key will fill out the command to be: copy running-config startup-config Best practice: backup copy of the running configuration To create a backup copy of the running configuration, use one of these methods: Copy the running-config or startup-config files to the switches flash directory
VDX# copy running-config flash://filename
Upload the startup-config or running-config files to a remote location using FTP or SCP, or locally using USB or flash
VDX# copy running-config ftp://login:password@IPaddress/path/filename
Restore configurations from backup copies: Download a remote configuration file by FTP or SCP Copy a file from the flash:// directory
VDX# copy flash://filename startup-config This operation will modify your startup configuration. Do you want to continue? [y/n]: y
49
Startup configuration file was copied successfully.
Examples: To copy the source file from a remote machine using SCP:
VDX# copy scp://user:password@10.10.10.10/path/filename startup-config
If a configuration name has an asterisk (*) appended to it if an outstanding transaction exists; the asterisk is not present if no outstanding transaction exists.
50
vLAG
A vLAG is a fabric service that allows LAGs to originate from multiple Brocade VDX switches acting as a single logical switch to an external switch or server. It acts the same way as a standard LAG using the Link Aggregation Control Protocol (LACP), a method to control the bundling of several physical ports together to form a single logical link or trunk. See Figure43.
Figure 43: vLAGs
vLAG Features Provisioning and management is consistent with a standard LAG implementation
Interoperable with servers and third-party switches - Standard LACP (IEEE 802.3ad) based interoperable solution Supports vLAG links across multiple Brocade VDX switches - NOS v2.1.x supports four Brocade VDX switches in a vLAG2 - They do not need to be directly connected - NOS v2.0.x only supports two Brocade VDX switches in a vLAG. From a user perspective, features running on top of the vLAG are configured and operate the same as
features running over a standard LAG (i.e. ACL, QoS)
NOS v2.0.x also supports a Brocade-proprietary aggregation in standalone mode only. NOS v2.1.x
supports the Brocade-proprietary aggregation in standalone and Brocade VCS mode.
Only ports with the same speed are aggregated. Brocade proprietary LAGs are not available for vLAGs. LACP automatically negotiates and forms the vLAG. A port-channel interface is created on all the vLAG members.
51
The Brocade VCS Fabric relies on you to consistently configure all nodes in the vLAG. Similar to static LAGs, vLAGs are not able to detect configuration errors. A zero port vLAG is allowed. IGMP snooping fits into the primary link of a vLAG to carry multicast traffic.
vLAG Example Topology Connection between the core Multi-Chassis Trunking (MCT) chassis and Ethernet fabric forms a single vLAG. Figure44 displays the following:
The two MCT chassis form a single logical device Brocade VDX switches form a single logical fabric Eliminates need to run STP between tiers (from access to core) Active-active redundant paths between the network tiers
Server connectivity into the fabric gain active-active link utilization. Brocade ISLs or trunks are formed within the Ethernet fabric. Multi-Chassis Trunking (MCT), currently available in Brocade MLX routers, is a Brocade technology that allows multiple switches to act as a single logical switch connected to another switch using a standard LAG. MCT at the core with vLAG in the edge fabric enables a single LAG between the two logical elements which results in a fully active-active network. Virtual LAG (vLAG): MCT is an industry accepted solution to avoid spanning tree in Layer 2 topologies. LAGbased MCT is a special case of LAG covered in IEEE 802.3ad in which the LAG ends terminate on two separate chassis. Virtual LAG (vLAG), a Brocade innovation, further extends the concept of LAG by allowing its formation across two or more physical switches.
52
Figure 44: vLAGs with MCTs
vLAG Provisioning Similar to LAG provisioning. Once the VCS fabric detects that the LAG configuration spans multiple switches, the LAG automatically becomes a vLAG. The standard Admin Key (Channel #) provisioned needs to be same for ports that belong to the same vLAG. Only ports matching the port-channel speed will be aggregated. See Figure45. vLAG formation: LACP will be used to automatically negotiate and form a vLAG with a server/switch. LACP on VCS fabric will emulate a single logical switch by Sending same LACP SYSTEM ID Sending same admin/oper key
53
A port-channel interface is created on all the vLAG participating members. LACP based vLAGs will detect partner mis-wiring scenarios (Remote SID will not match) For static vLAGs, admin is responsible to resolve partner mis-wiring All vLAGs rely on admin to configure consistent configuration (viz., switchport, vlan membership etc.) on all nodes in the vLAG.
Mode specifies the mode of Link Aggregation: Active enables the initiation of LACP negotiation on an interface. Passive enables LACP in passive mode, meaning the port will respond to received LACP frames but not initiate the LACP exchange On enables static link aggregation on an interface.
Figure 45: vLAG Provisioning
54
vLAG Optimized Multi-pathing Equal-cost multi-pathing demands that all the member RBridges in a vLAG are equal cost from a given ingress, otherwise multi-pathing is not exercised. Brocade VCS Fabric technology delivers an Optimized MultiPathing. Traffic is load shared across RBridges based on destination MAC addresses. The decision is made at the entrypoint of the fabric. See Figure46 Example, if a vLAG connection for Virtual Machine Server A was between RB2 and RB4, traffic destined different Virtual Machine (MACA and MACB) would be load share on an alternate path
Figure 46: vLAG Optimized Multi-pathing
LAG Type Brocade

A Brocade-proprietary aggregation1 is similar to standards-based link aggregation but differs in how the traffic is distributed. Utilizes frame-based load balancing to distribute traffic across the member link. For a LAG type of Brocade: All member links must be part of the same port-group Maximum of 8 ports in one LAG Maximum of 6 LAGs in one port group (6 x 2 port LAGs) Port Groups and the Brocade LAG: The Brocade VDX 6720-24 and VDX 6730-32 has two port groups; ports 1-12 and 13-24. The Brocade VDX 6720-60 and VDX 6730-76 has six port groups; ports 1-10, 11-20, 21-30, 31-40, 41-50 and 51-60.
55
The Brocade VDX 6710-54 port groups are: 1GE: 1-14, 15-27, 28-48 and 10GE 49-54 In NOS v2.1.x Brocade LAGs are supported in VCS Fabric mode; NOS v2.0.x and above supports standalone mode. This type of LAG could be used when connecting to a: Brocade 8000 switch or FCoE 10-24 blade Between two Brocade VDX switches where both are in standalone mode Between two Brocade VDX switches where one is in standalone mode and the other in VCS Fabric mode Between two Brocade VDX switches from different VCS fabrics
To create a LAG with type Brocade:

sw0(conf-if-te-32/0/12)# channel-group 20 mode active type brocade
To view avialable LAG types:

sw0(conf-if-te-32/0/12)# channel-group 20 mode active type ?
Possible completions:
brocade standard Brocade LAG Standards based LAG
56
Data Center Bridging (DCB)

Traditional Ethernet is not suitable for protocols that require or could benefit from a lossless, low latency and low congestion medium such as Fibre Channel or iSCSI. DCB is an umbrella term for an Ethernet technology enhanced by additional standards to meet these requirements. Also referred to as Converged Enhanced Ethernet (CEE). Key features of DCB include:
A lossless, full-duplex Ethernet environment which provides in-order delivery and supports a minimum of
2.5 KB mini-jumbo frames
Convergence of multiple protocols with different requirements over the same Ethernet network High speed transport of Ethernet traffic
Enhanced Ethernetalso called Converged Enhanced Ethernet (CEE), Data Center Ethernet or Data Center Bridging (DCB)eliminates Layer 3 TCP/IP protocols in favor of native Layer 2 Ethernet. Traditional Ethernet commonly experiences network congestion, latency and dropped frames, which renders it unreliable for Fibre Channel traffic. However, 10 GbE Enhanced Ethernet changes this by dispensing with TCP/IP in favor of a lossless Ethernet fabric. The lossless environments basic requirements are Priority Flow Control (priority pause), ETS (scheduler), and the discovery protocol. (Congestion management is attractive but optional.) These capabilities allow the Fibre Channel frames to run directly over 10Gbps Ethernet segments with no performance degradation. In order to make Ethernet as reliable as Fibre Channel at the lower layers, it is necessary to add several enhancements to Ethernet to mimic the behavior at the lower layers of Fibre Channel. For example, the PAUSE command is necessary to avoid congestion and frame drop, as is the case with a best-effort network such as Ethernet. There have to be logical channels on the Ethernet link, just as virtual channels exist in our Fibre Channel technology. This allows storage traffic to move reliably through the network, while IP traffic is handled on a best-effort basis. Fibre Channel FSPF allows multiple paths to be used between switches if their path cost is the same, but Ethernet with its Spanning Tree (and variations) routing model does not. Other enhancements allow for the detection and communication of congestion in the network, to avoid hot spots and throttle back inbound traffic. DCB Enhancements to Ethernet Data Center Bridging eXchange (DCBX) is based on IEEE 802.1Qaz and leverages functionality provided by IEEE 802.1AB (Link Layer Discovery Protocol - LLDP). It is used for conveying capabilities and configuration features between single-hop neighbors. Priority Flow Control (PFC) based on IEEE 802.1Qbb provide the following:
Link level flow control mechanism Controlled independently for each Class of Service (CoS). The Ethernet CoS is different from the Fibre
Channel CoS. It is defined as the priority level in DCB.
Goal of this mechanism is to ensure zero loss under congestion in DCB networks
Enhanced Transmission Selection (ETS) is based on IEEE 802.1Qaz and enables bandwidth management by assigning bandwidth segments to different traffic flows.
57
DCBX The Data Center Bridging Capability Exchange Protocol (DCBX) is used to exchange DCB-related parameters with neighbors including bandwidth allocation via ETS and flow control configuration via PFC. It is based on IEEE 802.1AB which defines a set of DCBX specific TLVs. DCBX capabilities include:
DCB peer discovery Mismatched configuration detection DCB-link peer configuration

DCBX uses LLDP to exchange parameters between two link peers. The DCBX TLVs are added to the LLDP frame. There are two types of LLDP TLVs, as specified in the IEEE 802.3AB standard:
Basic management TLVs consist of both optional general system information TLVs as well as mandatory
TLVs. Mandatory TLVs cannot be manually configured. They are always the first three TLVs in the LLDPDU, and are part of the packet header. General system information TLVs are optional in LLDP implementations and are defined by the Network Administrator. Common Basic Management TLVs include:
Chassis ID (mandatory) Port ID (mandatory) Time to Live (mandatory) DCBX DCBX Control Sub-TLV Priority group sub-TLV Priority flow control sub-TLV Logical link down feature sub-TLV Port description System name System description System capabilities Management address End of LLDPDU
Organizationally-specific TLVs are optional in LLDP implementations and are defined and encoded by
individual organizations or vendors. These TLVs include support for, but are not limited to, the IEEE 802.1 and 802.3 standards and the TIA-1057 standard. Examples of Organizationally-specific TLVs include:
802.1 organizationally-specific TLVs: 1) Port VLAN ID; and 2) VLAN name TLV
802.3 organizationally-specific TLVs: 1) MAC/PHY configuration/status; 2) Power through MDI; 3) Link aggregation; 4) Maximum frame size
58
ETS Overview Enhanced Transmission Selection (ETS) shown in Figure47 allocates bandwidth between different traffic classes such as: LAN FCoE iSCSI Inter-process communication (IPC) is normally server-to-server control traffic, and therefore has a very high priority.
Figure 47: ETS
ETS and Class of Service ETS uses Class of Service (CoS) to distinguish between traffic classes and allocate bandwidth to each traffic class. Allows bandwidth-allocated traffic classes to coexist with strict priorities traffic classes through the use of a hybrid scheduler. Only enforced when congestion is detected. See Figure48.
59
Figure 48: ETS and Class of Service
Strict Priority Scheduling A strict priority scheduler handles only very high priority traffic such as IPC traffic. Inter-process communication (IPC) is normally server-to-server control traffic, and therefore has a very high priority. Strict priority uses priority group ID 15. There are 8 levels within group 15, 15.0 15.7, with 15.0 being the highest priority and 15.7 being the lowest priority. Strict priority scheduling does not use any type of round robin servicing of queues and there is no bandwidth allocation. If there is more than one level of group 15 traffic, all of the highest priority traffic goes first, followed by the next highest priority For example, if there is traffic set for strict priority 15.1, and 15.2, all 15.1 traffic would be handled first, and then all of 15.2 traffic
60
FCoE and iSCSI

Physical port mapping:
<Switch_RBID>/<Slot_Number>/<Physical_Port>
For Brocade VDX departmental switches the slot number is always 0 Example: 10/0/24 would be switch RBID 10, slot 0 and port 24 FCoE port format
<Mapped-VN-Number>1/<Switch_RBID>/<Logical_Port>
The VN number is a virtual network number which is mapped to a VLAN (used for FCoE traffic). In this release, the VN number is fixed to 1 and the default VLAN used for FCoE traffic is 1002. The default VLAN number, 1002, can be changed. <Mapped-VN-Number> is fixed to 1 <Switch_RBID> is the switch ID of the Brocade VDX switch <Logical_Port> is the logical port number Example: Interface FCoE 1/10/24 would be VN 1, switch RBID 10 and port 24 1:1 Mapping of physical port to FCoE logical port Example: TenGigabitEthernet 10/0/24 = FCoE 1/10/24 where port 24 matches FCoE Co-Existence with LAG/vLAG FCoE provisioning is allowed on LAG members. One or more members of a LAG can be provisioned for FCoE. FCoE provisioning not allowed on the LAG itself and there is no spraying/load-balancing of FCoE traffic. Only supported between a host and switch and requires NOS v2.1.0. FCoE traffic is not allowed outside the fabric. See Figure49. Note: The LAG hash algorithms will not be used for FCoE load balancing. host MPIO drivers will be used to load balance traffic if 2 or more physical ports in the LAG are enabled for FCoE.
Figure 49: FCoE Co-Existence with LAG/vLAG
61
Configuring FCoE Configures and enables FCoE on the physical port. Use the FCoE map named default which contains: CEE map configuration Fabric map configuration If no changes to any of the defaults are required then the only command needed is: fcoeport default which enables FCoE and applies the CEE and fabric maps
RB1# configure terminal RB1(config)# interface TenGigabitEthernet 10/0/24 RB1(config-TenGigabitEthernet-10/0/24)# fcoeport default
The At this point the attached FCoE device should login The CEE map defines the traffic type to a CoS and defines the allocated bandwidth for each traffic type. The Fabric map defines the: (defaults) which are:
FCoE VLAN (1002) FCoE CoS (3) FCMAP used for FPMA addressing (0E:FC:00) Keep-alive timeout value (enabled) Advertisement interval (8 seconds)
Configuring FCoE and LLDP Verify the LLDP advertisements in the running-config:
RB1# show running-config protocol lldp protocol lldp advertise dcbx-fcoe-app-tlv advertise dcbx-fcoe-logical-link-tlv advertise dcbx-tlv
Enable LLDP FCoE TLV1 advertisement
Notes:
DCBX FCoE application TLV which advertises the DCBX2 FCoE application TLV DCBX FCoE logical link TLV which advertises the DCBX FCoE logical link TLV3 The DCBX-TLV is advertised by default which means the switch is DCB4 capable
TLV is Time Length Value. The TLV provides the status of the FCoE link. The FCoE link must be up before the attached FCoE device can send a FIP frame. After all changes are made to enable the physical port for FCoE, you use:
62
RB1# configure terminal RB1(config)# interface tengigabitethernet 10/0/24 RB1(config-TenGigabitEthernet-10/0/24)# fcoeport default
At this point the FCoE device should login. The command fcoeport default provides the FCoE which contains the CEE and fabric maps Enable and Disable FCoE Ports FCoE logical ports are enabled (no shutdown) by default. The configuration commands are: Shutdown (disable) No shutdown (enable) Example where 1/10/24 is the FCoE logical port:
RB1# configure RB1(config)# interface fcoe 1/10/24 RB1(config-Fcoe-1/10/24)# shutdown RB1(config-Fcoe-1/10/24)# no shutdown
This is a good way to bounce the FCoE logical link but keep the non-FCoE traffic flowing FCoE show Commands Display interface information in brief. See Figure50. Shows the status of the protocol: Is the switch and attached device exchanging frames and displays the number of VN ports for each port. Here are details: FCOE IF is the FCoE logical port. Mode Config/Current: In this release, this always show VF (Virtual Fabrics). Virtual Fabrics is not supported in this release, the switch is always in the default fabric. This is setup for future enhancements. Status Config is the status (enabled or disabled) of the logical FCoE port. Status Proto displays the status of the protocol. Are the two devices talking to each other?
This does not mean that all is OK, the configuration could still have problems, but is still sending frames back and fourth. If the Protocol is up but the VN Port count is 0, check the configuration of the physical port.
Binding displays the physical port where the FCoE logical port is mapped to. Num VN Ports displays the number of devices that have successfully logged into the fabric. Note: If the Config status shows as Admin-Down that mean the FCoE logical port is shutdown.
63
Figure 50: show fcoe interface brief command example
iSCI TLV Support DCB support for the iSCSI TLV1 provides deterministic delivery of iSCSI traffic. Where DCB support is required for FCoE, iSCSI does not require DCB capabilities. That said there are benefits to using it. Such as the ability to pause traffic and the allocation of bandwidth. Intent is to minimize TCP retransmissions by eliminating packet loss do to congestion. Supports DCBX, PFC and ETS. The DCB iSCSI TLV is supported in both standalone and VCS Fabric modes. NOS version 2.0.0a supported the TLV in standalone mode only. Supported on all the Brocade VDX 10 GbE ports Brocade VDX 6710: 1 GbE ports do support iSCSI but since the ports are not DCB capable they do not support the iSCSI TLV
64
Zoning
FCoE only zoning requires Brocade VDX switches to be running NOS v2.1.0 or higher For FCoE to FC Routing requires: Brocade VDX 6730 switches to be NOS v2.1.1 Brocade SAN backbone fabric to be running FOS v7.0.1 Zone members are WWN only (port or node). Port-based zones, i.e., "Domain,Index" format as a zone member, is not supported For LSAN zones, only port WWNs can be used VCS Fabric mode must be enabled Zone enforcement is done by Name Server Meaning it is soft zoning For Fibre Channel zoning, there are two types of zoning: Session-based hardware enforcement Name Server restricts PLOGIs Frame-based hardware enforcement Source device is denied access to destination device if they are not defined in the same zone Available through ASIC hardware logic checking at the destination port More secure than session enforcement
Zone Merging
When a new switch is added to the fabric, it automatically takes on the zone configuration information from the fabric. If a switch is being added that is already configured for zoning, clear the zone configuration on that switch before connecting it to the zoned fabric. If default zoning is implemented, must set the switch being added to the fabric to the same default zone mode setting If settings do not match the new switch will segmented
65
FC Bridging
Brocade VCS Fabric and Brocade Fibre Channel SAN connectivity is enabled through FC-FC Routing. See Figure51. Connectivity is provided through an E_Port on a Brocade VDX 6730 which connects to an EX_Port on a FC Router
Figure 51: Brocade VCS Fabric to Brocade FC SAN Connectivity
FC-FC Routing to Bridge FCoE and FC Traffic Fabric OS provides Layer 3 Fibre Channel-to-Fibre Channel routing (FC-FC Routing) between fabrics. Allows device access between two or more fabrics without merging the fabrics. Brocade VCS Fabrics use the FC-FC routing technology to bridge traffic between Brocade VCS Fabric FCoE devices and Brocade FOS FC targets.
66
FC to FC Routing Physical connectivity is accomplished through the use of a Fibre Channel Router (FCR).Includes implementing and configuring the underlying physical connectivity between the fabrics that will share devices using EX_Ports/Inter Fabric Links (IFL). See Figure52. FC-FC routing was introduced in Fabric OS v5.1 on the Brocade 7500 and Brocade FR4-18i blade. FC-FC routing is also known as, FCRS (Fibre Channel Routing Service), FC-to-FC routing, FCR, FC routing and routed SANs. The FC router in effect enforces an implied DENY_ALL, and the administrator must configure the PERMIT entries (ACLs) via LSAN zoning. Logical connectivity is accomplished through the use of Logical Storage Area Networks (LSANs), by creating uniquely named zones called LSAN zones.
Figure 52: FC to FC Routing
67
Brocade VDX Switch TRILL Termination

FD and XD domains are represented as RBridges in the NOS fabric. The front domain and xlate domains count against the 24 RBridge fabric limit. See Figure53. Brocade VDX 6730 switch provides TRILL path termination for FC domains Proxy RBridge ID Proxy FCF MAC
Figure 53: Brocade VDX Switch Trill Termination
68
Edge to BB Configuration
From the FCR, configure an EX_Port to communicate with the Brocade VCS fabric. See Figure54. Disable the port that you wish to connect to the Brocade VDX switch FCR:admin> portdisable x To configure and activate the EX_Port on port 0 of the FCR, with the Interopmode set to 5 (Brocade VCS Fabric), issue command: FCR:admin> portcfgexport 0 a 1 m 5
Figure 54: Edge to BB Configuration
69
AMPP
VM Mobility Challenges Virtual machine mobility (migration) challenge is when server managers, like VMwares vCenter, control the migration of server side profiles to ensure the server-side consistency. See Figure55. In traditional networks, post-migration tasks often require manual configuration changes. VM migration across physical server and switches can result in non-symmetrical network policies. Port setting information must be identical at destination switch/port.Brocade AMPP technology enhances network-side Virtual Machine migration by allowing VM migration across physical switches, switch ports, and collision domains.
Figure 55: VM Migration Challenges
Solution Overview Deploy and distribute port profiles within the Ethernet fabric. VMs can be migrated without a need for network ports to be manually configured on the destination switch. Brocade VCS Fabrics support automatically moving the port profile in synchronization with a VM moving to a different physical server on the same or different access layer switches. Allows for rapid migration of applications in the data center. SeeFigure56. AMPP belongs under the IEEE 802.1 work group and the emerging standard called Edge Virtual Bridging (EVB)
70
Figure 56: Port Profiles: Distributed Network Settings
Port Profile Overview A port profile contains the entire configuration needed for a VM to gain access to the LAN and FCoE The contents of port profiles can be any of the following: Only LAN configuration Only FCoE configuration Both FCoE and LAN configurations A port profile does not contain some of the interface level configurations such as LLDP, SPAN, and LAG. A port profile is capable of operating as a self-contained configuration container. In other words, if a port profile is applied on a new switch without any configuration, a port profile should be sufficient enough to start carrying traffic.
71
Figure 57: Port Profiles
For normal AMPP operation, port profiles need to be pre-created manually across fabric. Manual configuration of ports/LAG/vLAG in port-profile mode across the fabric. NOS v2.1.0 VMware vCenter integration provides the ability to: Automatically create AMPP port-profiles from VM port groups Automatically create VLANs Automatically create association of VMs to port groups Automatically configure port-profile modes on ports
NOS v2.1.0 supports vCenter version 4.0 or greater NOS v2.1.1 is required to support vCenter 5.0
72
vCenter Integration Implementation

Brocade VDX switch is configured with vCenter access information and credentials (URL, user name and password). Brocade VDX switch performs discovery of virtual infrastructure assets. Based on discovered assets, the VCS fabric will automatically configure corresponding objects: Port-profiles and VLAN creation MAC address association to port-profiles Port, LAGs, vLAGs are put into profile mode automatically based on ESX host connectivity The VCS fabric is ready for Virtual Machine movements NOS vCenter Integration Each RBridge in the fabric listens for CDP1 (Cisco Discovery Packet) packets from ESX hosts on switch ports. Automatically ports/LAGs/vLAGs are put in profile mode when the connected ESX host transmits CDP. Removes the port from profile mode in the case that CDP is timed out on that port. vCenter uses CDP so Brocade VDX switches have to support this. The VMware vSwitch or dvSwitch must be configured for CDP. CDP is configured through vCenter using the Properties dialog of the logical switch. See Figure58.
Figure 58: NOS and vCenter Integration
73
General Ingress Traffic Scenarios A Port Profile Created with these features. See Figure59. VLAN2 (tagged). VM1 is not permitted to access MAC3. VM1, VM2, VM3 have been associated with their MAC within this port profile. Incoming Traffic (Ingress Behavior): VM1 enters in VLAN2 and is trying to access VM2 in VLAN2. This is permitted because its MAC is on a permitted this VLAN and our policy only blocks VM1 access to VM3. VM1 enters in VLAN2 and is trying to access VM3 in VLAN2. This is not permitted per our defined ACL. VM2 enters in VLAN3 and is trying to access any other VM. This not permitted as not defined in our port profile. VM1 enters in VLAN2 is coming in on a non Port-Profile Port. In this release this traffic is permitted, but a future requirement will make all VM traffic already associated with a port profile to be on a port which has a port-profile enabled. VM7 enters into a port with its MAC not associated to a Port-Profile. The traffic is blocked.
74
Figure 59: Ingress Traffic Example
75
Management and Troubleshooting
sFlow Overview sFlow is an industry standard for monitoring high-speed, multilayer switched networks and complies with RFC 3176. It monitors traffic flow for all network ports or specific ports by using a sampling technology to collect statistics. It is effective at gigabit speeds and does not impact network performance. sFlow is protocol independent to ensure that all the traffic is seen and is supported on physical interfaces only.
Benefits of sFlow: - Minimizes network downtime - Rapidly pin-points congestion problems - Troubleshoots network problems quickly - Detailed data capture enables rapid problem resolution, minimizing costly network - Protect the assets with Security and Surveillance - Identify access policy violations and intrusions - Distributed Denial of Service Detection and diagnosis
sFlow Components sFlow uses two types of sampling:
Statistical packet based sampling of packet flows (this release samples packet headers only)
Time based sampling of interface counters (this release samples generic and Ethernet counters) sFlow Protocol Overview sFlow monitors high-speed switched networks. The sFlow agent collects statistics from the switch and forwards the data to the sFlow collector. The sFlow collector stores the sFlow datagrams from all agents on the network for processing at a later time. The sFlow agent uses two forms of operation:
Time-based sampling of interface counters Statistical sampling of switched packets

sFow Deployment Measures for any and every port and is a real time at central collector. See Figure60. The sFlow subsystem consists of: Data source is the port on a switch. sFlow agent (sflowd). sFlow collector is the host to which sample data is sent.
76
Figure 60: sFlow Deployment
sFlow Global Configuration Enable sFlow

RB1(config)# sflow enable
Configure sample rate. sFlow sample Rate: Default is 32768, range is from 2 1677721. A flow sample is based on random packets being forwarded to the sFlow collector at defined numeric intervals, either for the entire Brocade switch or for a single port interface. For example, every 4,096th packet is forwarded to the sFlow collector for analysis and storage.
RB1(config)# sflow sample-rate 512
Configure polling interval (In seconds). sFlow polling interval: Default is 20, range is from 1 40.A polling interval defines how often the sFlow octet and packet counter for a specific interface are sent to the collector, but the sFlow agent is free to schedule the polling in order to maximize internal efficiency. If the regular schedule is chosen, each counter start time will be chosen randomly to eliminate bottlenecks in network performance.
RB1(config)# sflow polling-interval 20
77
Configure sFlow collector

RB1(config)# sflow collector 192.168.10.49
Putting a no in front of the command will remove the configuration. Brocade recommends that you globally configure sFlow on the switch first, then enable sFlow on specific interface ports and make custom alterations. Switched Port Analyzer SPAN (Switched Port Analyzer) feature copies traffic from a port and sends it to a SPAN port. Also known as Port Mirroring. Where an analyzer can be connected to capture/analyze traffic. Could also be a host running Wireshark or a similar Ethernet traffic capturing tool. See Figure61. Is a debugging feature which can work without disturbing existing connections (No down time) Note: Using a capture tool instead of an analyzer can result in gaps in the captured data and the capture has to stop in order to dump its data to a capture file. Note: JDSU Xgig TraceView software 5.x supports the decode of TRILL fames. NOS v2.1.0: The source port can be a port-profile port. A maximum of 24 mirror sessions are supported in StandAlone and Fabric mode. A maximum of 512 sessions are supported in Management Cluster mode.
Figure 61: SPAN
SPAN Limitations Source and destination interfaces MUST be on the same eAnvil2 ASIC Brocade VDX 6720-24, 6730-32: Any two ports
78
Only one port can be configured as a destination port Brocade VDX 6720-60, 6730-76 Ports 1-10, 11-20, 21-30, 31-40, 41-50 and 51-60 Only one port per port group (ASIC) can be configured as a destination port Trill (ISL) ports can not be source or destination ports. Port-channels (LAGs/vLAGs) can not be source ports. Physical ports belonging to a port-channel can be configured as a source port. FCoE logical ports are not supported.
Collecting Support Data

If escalation is required capture one of the following: From the CLI run the copy support command for every switch in the fabric From Brocade Network Advisor capture a supportsave from every switch in the fabric
Security
Login Authentication Authentication mode is defined as the order of authentication sources to be used for user authentication (the login process). Two sources of authentication are supported: primary and secondary. The secondary source of authentication is used in the event of primary source failover and is optional for configuration. There are three possible sources: Local (the default source) RADIUS TACACS+ TACACS+ Overview The Terminal Access Controller Access-Control System Plus (TACACS+) is a protocol used in AAA server environments. It is used only for authentication. A maximum of five TACACS+ servers can be configured per switch. A role should be assigned to a user configured on the TACACS+ server and configured on the switch. The user role is assigned by default when the following occur:
If the switch fails to get the users role from the TACACS+ server after successful authentication If the role does not match any of the roles present on the switch
RBAC Overview Role-based access control (RBAC) is used as an authorization mechanism. Roles can be created dynamically. Roles are associated with rules to define permissions (i.e. read-only, read-write). User accounts must be associated with only one role. Permissions cannot be assigned directly to the user accounts. Permissions can
79
only be acquired through the associated role. RBAC is the function of specifying access rights to resources for roles. When a user executes a command, privileges are evaluated to determine access to the command based on the role of the user. Rules can be created for the specified operational commands. By default, every role can display all the operational commands but cannot execute them. The show commands can be accessed by all the roles. The following rules govern operational commands: If a role has a rule with a read-write operation and the accept action for an operational command, the user associated with this role can execute the command. If a role has a rule with a read-only operation and the accept action for an operational command, the user associated with this role can access but cannot execute the command. If a role has a rule with a read-write operation and the reject action for an operational command, the user associated with this role can neither access nor execute the command. SCC Policy Overview The Switch Connection Control (SCC) policy controls how neighboring switches connect to the switch. The switch connection control (SCC) policy is used to restrict which switches can join the fabric by either accepting or rejecting the connection between two switches. Switches are checked against the policy each time an E_Port-to-EX_Port connection is made. The policy is named SCC_POLICY and accepts members listed as WWNs. ACL Overview ACLs filter traffic for the Brocade VDX hardware and permit or deny frames on ingress interfaces that have the ACLs applied to them. You can apply ACLs on the three kinds of Layer 2 interfaces that Brocade Network OS v 2.1.1 supports; physical (both tengigabitethernet and gigabitethernet), VLAN, and port-channel (both static and dynamic LAG). Each ACL is a unique collection of permit and deny statements (rules) that apply to frames. When a frame is received on an interface, the switch compares the fields in the frame against any ACLs applied to the interface to verify that the frame has the required permissions to be forwarded. The switch compares the frame, sequentially, against each rule in the ACL and either forwards the frame or drops the frame. A security profile defines all the security rules needed for the server port. A typical security profile contains attributes for MAC-based standard and extended ACLs. MAC ACLs are supported on the following interface types and do not take effect until applied to a Layer 2 interface: Physical interfaces Logical interfaces (LAGs) VLANs
80
VDX Additional Functionality

Brocade NOS supports two modes of operation for Brocade VDX platforms: Brocade VCS Fabric technology mode
Ethernet Fabric FCoE DCB capable with iSCSI support VM-aware networking Functions as a standard 802.3 switch Uses Spanning Tree to prevent loops (no Ethernet Fabric support) DCB capable with iSCSI support No FCoE support. End to End FCoE is only supported in Brocade VCS Fabric mode.
Standalone mode
Here are the features available in both standalone and Brocade VCS fabric technology mode: Layer 2 data forwarding MAC learning and aging BPDU Drop PING and Trace Route STP, Rapid Spanning Tree Protocol (RSTP), Multiple Spanning Tree Protocol (MSTP) Per VLAN Spanning Tree Plus (PVST+), Per VLAN Rapid Spanning Tree Plus (PVRST+) [ LACP, Brocade ISL Trunking Link-Level Discovery Protocol (LLDP) and Data Center Bridging Exchange (DCBX) IEEE 802.1x sFlow Switched Port Analyzer (SPAN) Layer 2 access control lists (ACLs) Management port IP ACL (standard and extended) SNMP NETCONF support (RFC 4741) LDAP v3 (RFC 4510) Enhanced Transmission Selection (802.1Qaz) Priority-based flow control (802.1Qbb) Internet Group Management Protocol (IGMP) snooping Automatic Migration of Port Profiles (AMPP) In-band management TACACS+ Internet Small Computer Systems Interface (iSCSI) DCBX support
81
Here are features available only in Brocade VCS fabric technology mode: Transparent LAN service Virtual link aggregation groups (vLAGs) Distributed configuration management End-to-end FCoE Fibre Channel fabric connectivity VM-aware networking
Port Configuration Difference for Standalone Mode

Ports on a switch in standalone mode are referenced as follows:
switch(config)# interface TenGigabitEthernet 0/1
In the above example: Zero represents the slot number which for non-chassis-based systems this is always zero One represents the physical port number For a switch with Brocade VCS Fabric technology enabled, the RBridge id is always added first as shown here: 3/0/1. The other information remains the same.
Trunk Mode Native VLAN

A Native VLAN can be defined as a VLAN that is not associated with any tag on a 802.1Q link. The Native VLAN is used to manage all untagged traffic received on a 802.1Q port. The concept of a trunk port which uses Native VLAN is that once a port is designated as a trunk port, the trunk port will forward and receive tagged frames. Frames belonging to the native VLAN do NOT carry VLAN tags when sent over the trunk. If an untagged frame is received on a trunk port, the frame is associated with the Native VLAN for the 802.1Q port. By default, VLAN 1 is designated as the Native VLAN In Clause 9 of the 1998 802.1Q standard, the standard defines the encapsulation protocol used to multiplex VLANs over a single link by adding VLAN tags. However, it is possible to send frames either tagged or untagged (no VLAN tags). To support both tagged or untagged frames, vendors created the concept a trunk port that uses the Native VLAN to support the tagged or untagged frames. The concept of a trunk port is that once a port is designated as a trunk port, it will forward and receive tagged frames. Frames belonging to the native VLAN do not provide VLAN tags when sent over the trunk. Conversely, if an untagged frame is received on a trunk port, the frame is associated to the Native VLAN for this port.
82
For example, if an 802.1Q port has VLANs 2 and 3 assigned to it with VLAN 2 being the Native VLAN, frames on VLAN 2 that egress (exit) from that port are not given an 802.1Q header as the frame are plain Ethernet frames. Frames which ingress (enter) this port and have no 802.1Q header are put into VLAN 2, the Native VLAN. For VLAN 3 the frames arriving for VLANs 3 are expected to contain tags that identify as VLAN 3 and frames leaving the port for VLANs 3 carry the VLAN tag information. VDX Hardware and Ethernet Frames When the destination MAC address is not in the lookup table, the frame is flooded on all ports in the same VLAN, except the ingress port. When the destination MAC address is present in the lookup table, the frame is switched only to the correct egress port. When the destination MAC address is present in the lookup table, and the egress port is the same as the ingress port, the frame is dropped. If the Ethernet Frame Check Sequence (FCS) is incorrect, because the switch is in cut-through mode, a correctly formatted Ethernet frame is sent out with an incorrect FCS. If the Ethernet frame is too short, the frame is discarded and the error counter is incremented. If the Ethernet frame is too long, the frame is truncated and the error counter is incremented. The truncated frame is sent out with an incorrect FCS. Frames sent to a broadcast destination MAC address are flooded on all ports in the same VLAN, except the ingress port. When MAC address entries in the lookup table time out, they are removed. In this event, frame forwarding changes from unicast to flood. An existing MAC address entry in the lookup table is discarded when a device is moved to a new location. When a device is moved, the ingress frame from the new port causes the old lookup table entry to be discarded and the new entry to be inserted into the lookup table. Frame forwarding remains unicast to the new port. When the lookup table is full, new entries replace the oldest MAC addresses after the oldest MAC addresses reach a certain age and time out. MAC addresses that still have traffic running are not timed out. New entries start replacing older entries when the lookup table reaches 90 percent of its 32K capacity.
Virtual IP Address
Allows administrator to access a VCS fabric using one virtual IP address. Not available in Standalone Mode: Standalone Mode is one VCS Fabric disabled switch. One VCS Fabric enabled switch still constitutes a VCS Fabric. The Brocade VCS Fabric can be assigned an IP address, known as a Virtual IP (VIP) address. When the VIP is used to connect to the fabric, successful authentication will log in the user into the Coordinator (principle switch) of the fabric.
83
By default, a virtual IP address (VIP) is not set, once set, it is always bound to the Coordinator. When the Coordinator fails over, it will be bound to the new Coordinator (Principal Switch). When two fabrics are merged, the new virtual IP address is the one that was set for the new Coordinator (new Principal Switch). When a fabric is segmented, both fabrics retain the virtual IP address. When accessing the coordinator using the VIP any changes to the running configuration will automatically get copied to the startup configuration. The VIP can be used to manage vCenter Integration, BNA Discovery, Zoning and management of the coordinator switch.
84
Taking the Test

After the Introduction Screen, once you click on Next, you will see the following non-disclosure agreement:
IMPORTANT: PLEASE READ THE FOLLOWING BROCADE NON-DISCLOSURE CONFIDENTIALITY AGREEMENT CAREFULLY BEFORE TAKING THIS EXAM.
The following Non-Disclosure Confidentiality Agreement (the Agreement) sets forth the terms and conditions of your use of the exam materials as defined below. The Disclosure to you of this Exam and any questions, answers, worksheets, computations, drawings, diagrams, or any communications, including verbal communication by any party, regarding or related to the Exam and such Exam Materials and any derivatives thereof is subject to the Terms and Conditions of this Agreement. You understand, acknowledge and agree:
That the questions and answers of the Exam are the exclusive and confidential property of Brocade and are
protected by Brocade intellectual property rights;
That you may not disclose the Exam questions or answers or discuss any of the content of the Exam Materials
with any person, without prior approval from Brocade;
Not to copy or attempt to make copies (written, photocopied, or otherwise) of any Exam Material, including,
without limitation, any Exam questions or answers;
Not to sell, license, distribute, or give away the Exam Materials, questions, or answers; You have not purchased, solicited or used unauthorized (non-Brocade sanctioned) Exam Materials, questions, or
answers in preparation for this exam;
That your obligations under this Agreement shall continue in effect after the Exam and, if applicable, after
termination of your credential, regardless of the reason or reasons for terminations, and whether such termination is voluntary or involuntary.
Brocade reserves the right to take all appropriate actions to remedy or prevent disclosure or misuse, including, without limitation, obtaining an immediate injunction. Brocade reserves the right to validate all results and take any appropriate actions as needed. Brocade also reserves the right to use any technologies and methods for verifying the identity of candidates. Such technology may include, without limitation, personally identifiable information, challenge questions, identification numbers, photographic information, and other measures to protect against fraud and abuse. Neither this Agreement nor any right granted hereunder shall be assignable or otherwise transferable by you. By clicking on the "A" button (YES, I AGREE), you are consenting to be bound by the terms and conditions of this agreement and state that you have read this agreement carefully and you understand and accept the obligations which it imposes without reservation. You further state that no promises or representations have been made to induce agreement and that you accept this agreement voluntarily and freely. A. B. YES, I AGREE NO, I DO NOT AGREE
85
86

Bcefe Nutshell

Cargado por

Información del documento

Descripción original:

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

Bcefe Nutshell

Cargado por

Copyright:

Formatos disponibles

BCEFE in a Nutshell Study Guide for Exam 150-620

Global Education Services Revision 0312

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

BCEFE in a Nutshell Second Edition

Joe Cannata Certification Manager

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell First Edition

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

Brocade VCS Fabric Technology

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

No single point failure No passive connections STP not necessary

Figure 1: VCS Fabric Use Case: Aggregation Layer

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

Figure 2: VCS Fabric Use Case: Combined Access/Aggregation Layers

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

Hardware Products and Features

1U 6 x 10 GbE SFP+ and 48 x 1 GbE GE copper

2U 24 x 10 GbE SFP+ ports 60 x 10 GbE SFP+ ports

Brocade VCS Fabric License

POD License Brocade VCS Fabric License FCoE License

High Performance, low latency enabled convergence switch with FC connectivity

Brocade VDX Switch Protocol and Fabric Support

Ethernet Yes Yes Yes

FCoE No Yes Yes

VCS Fabric Yes 3 Yes Yes

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

Brocade VDX 6720 Data Center Switches Overview

Figure 3: Brocades VDX 6720

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

Figure 4: Brocade VDX 6720-24 Data Flow

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

Figure 5: Brocade VDX 6720-24 Port Groups

Brocade VDX 6720-60 Data Flow

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

Figure 6: Brocade VDX 6720-60 Data Flow

2012 Brocade Communications

Brocade Certified Ethernet Fabric Engineer in a Nutshell Second Edition

Figure 7: Brocade VDX 6720-60 Port Groups

Brocade VDX 6710-54 Data Center Switch